{"note":"The data audit of every benchmark: how well we verified its record, who has scored it and how recently, how fresh our read of its leaderboard is, and how far labs' own numbers sit from independent runs. Each result is [check, status, detail]. A note (info) means the check does not apply or needs no fix.","asOf":"2026-11-21","statuses":{"pass":"Pass","warn":"Warning","fail":"Problem","info":"Note"},"groups":[{"id":"record","name":"Catalog record"},{"id":"scores","name":"Scores"},{"id":"import","name":"Leaderboard import"},{"id":"consistency","name":"Consistency"}],"checks":[{"key":"record.access","group":"record","public":true,"label":"Task access","problem":"unknown task access","rule":"Passes when the record says whether the tasks are public or private."},{"key":"record.confidence","group":"record","public":true,"label":"Confidence","problem":"a record we haven’t fully verified","rule":"Passes when we checked the record against a primary source. Likely means some facts rest on secondary sources. Unverified means we haven’t checked it yet."},{"key":"scores.sources","group":"scores","public":true,"label":"Independent runs","problem":"no independent run","rule":"Passes when someone other than the model’s maker measured scores: the official leaderboard, Epoch AI, or another board Epoch AI copied. An in-house eval can only have its lab’s numbers."},{"key":"scores.models","group":"scores","public":true,"label":"Models scored","problem":"fewer than five models","rule":"Passes with five or more models, enough to rank them."},{"key":"scores.recent","group":"scores","public":true,"label":"Newest score","problem":"no new score in six months","rule":"Passes when a model got a score in the last six months. A test that saturated or was replaced can go quiet."},{"key":"scores.frontier","group":"scores","public":true,"label":"Frontier models","problem":"few frontier models tested","rule":"Passes when at least three of the ten strongest models on Epoch AI’s Capabilities Index have a score here. A retired test or an in-house eval doesn’t need them."},{"key":"scores.headroom","group":"scores","public":true,"label":"Headroom","problem":"a saturated test","rule":"Passes while the best score on a percent test is under 85%. Past that, we count it as saturated, which is a note, not a problem."},{"key":"import.importer","group":"import","public":true,"label":"Importer","problem":"a board we don’t read yet","rule":"Passes when we read the official leaderboard ourselves. A test with no public leaderboard doesn’t need one."},{"key":"import.read","group":"import","public":true,"label":"Last read","problem":"a stale read","rule":"Passes when we read the board in the week before the data’s date."},{"key":"import.failed","group":"import","public":true,"label":"Read worked","problem":"a failed leaderboard read","rule":"Fails when the last attempt to read the board failed. Pages keep the rows from the last good read."},{"key":"consistency.claims","group":"consistency","public":true,"label":"Lab claims","problem":"lab numbers far from independent runs","rule":"Warns when a lab’s number for its own model is more than 10 points (or 10%) from an independent run of the same model, and we have no note on why."},{"key":"consistency.issues","group":"consistency","public":true,"label":"Known issues","problem":"an open audit finding","rule":"Fails while a published audit’s finding is still open. Disputed findings and fixes made only in a copy get a warning."}],"totals":[{"key":"record.access","pass":6437,"warn":565,"fail":0,"info":0},{"key":"record.confidence","pass":6448,"warn":542,"fail":12,"info":0},{"key":"scores.sources","pass":4777,"warn":1310,"fail":0,"info":915},{"key":"scores.models","pass":4232,"warn":0,"fail":0,"info":2770},{"key":"scores.recent","pass":2346,"warn":1716,"fail":0,"info":2940},{"key":"scores.frontier","pass":674,"warn":0,"fail":0,"info":6328},{"key":"scores.headroom","pass":2175,"warn":0,"fail":0,"info":4827},{"key":"import.importer","pass":4766,"warn":133,"fail":0,"info":2103},{"key":"import.read","pass":4762,"warn":4,"fail":0,"info":0},{"key":"import.failed","pass":4756,"warn":0,"fail":10,"info":0},{"key":"consistency.claims","pass":287,"warn":26,"fail":0,"info":6689},{"key":"consistency.issues","pass":6995,"warn":4,"fail":3,"info":0}],"benchmarks":[{"id":"onemillion-bench","name":"$OneMillion-Bench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Fable 5.1."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","pass","The best is 68.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on xbench.org: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 2 pairs differs by more than 10 points, the largest for Seed2.1 Pro (+45 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-forensics-selected-moderation-api","name":"(S)elected Moderation campaign prompts","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aiforensics.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"100q-hard-anthropic","name":"100Q-Hard (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (7 scores)."],["scores.models","pass","7 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","pass","The best is 60.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"1h-videoqa-google","name":"1H-VideoQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries) and system cards (2 scores)."],["scores.models","pass","21 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 89.1%, past 85% since August 2026."],["import.importer","pass","We read the board on kaggle.com: 20 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"1x-world-model-challenge","name":"1X World Model Challenge","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"i2b2-va-2010-concepts-assertions-relations","name":"2010 i2b2/VA Challenge on Concepts, Assertions, and Relations in Clinical Text","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nli-shared-task-2013","name":"2013 Native Language Identification Shared Task","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"2018-n2c2-cohort-selection-for-clinical-trials","name":"2018 n2c2 Cohort Selection for Clinical Trials","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"n2c2-2022-social-determinants-of-health","name":"2022 n2c2/UW Shared Task on Extracting Social Determinants of Health","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nist-genai-2024-t2t-discriminators","name":"2024 NIST GenAI Text-to-Text Discriminators","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (328 entries)."],["scores.models","pass","328 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on miganalytics.nist.gov: 328 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nist-genai-2024-t2t-generators","name":"2024 NIST GenAI Text-to-Text Generators","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a Brier score, with no fixed ceiling."],["import.importer","warn","The record links a board on miganalytics.nist.gov that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"times-crossword-championship-2026","name":"2026 Times Crossword Championship","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since June 2026."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-24-7-clawbench-2-moonshot","name":"24/7 ClawBench 2.0","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 48.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"japanese-dental-hygienist-exam-2025","name":"34th Japanese National Examination for Dental Hygienists","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"3d-animation-arena","name":"3D Animation Arena","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"3d-arena","name":"3D Arena","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","28 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 28 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"3d-gamedev-arena","name":"3D GameDev Arena","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 99.8%, past 85% since June 2026."],["import.importer","pass","We read the board on 110.238.108.113: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"subwaybench-command-code","name":"3D SubwayBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"3d-worlds-arena","name":"3D Worlds Arena","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on 3d-arena-3d-world-arena.hf.space: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"3darena-modeling-tencent","name":"3DArena-Modeling","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"3dcodearena","name":"3DCodeArena","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on 3dcodebench.com: 14 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"3dcodebench","name":"3DCodeBench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from November 2026, for LongCat-2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","pass","The best is 55.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"3dgen-bench","name":"3DGen-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 20 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"3dpw","name":"3DPW","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"3dscenebench","name":"3DSceneBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on pranalabs.github.io: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"3dsrbench","name":"3DSRBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (8 entries) and launch posts (1 score)."],["scores.models","pass","5 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for SenseNova-SI-1.3-InternVL3-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o mini (#178)."],["scores.headroom","pass","The best is 62.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on 3dsrbench.github.io: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"3lm","name":"3LM","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (48 entries)."],["scores.models","pass","25 models we track, plus 23 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Qwen3-30B-A3B-Base. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3-235B-A22B (#131)."],["scores.headroom","info","Saturated. The best is 85.7%, past 85% since December 2024."],["import.importer","pass","We read the board on aclanthology.org: 25 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"3sat-search","name":"3SAT-Search","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries) and system cards (2 scores)."],["scores.models","pass","13 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Phi-4-Reasoning-plus. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1 (#118)."],["scores.headroom","info","Saturated. The best is 96.2%, past 85% since December 2024."],["import.importer","pass","We read the board on huggingface.co: 11 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"4dcodebench","name":"4DCodeBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on 4dcodebench.com: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"67ai-l2-report-structuring","name":"67AI Lab L2 nuclear medicine report structuring","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on 67ailab.com: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"67ai-l3-guideline-qa","name":"67AI Lab L3 nuclear medicine guideline questions","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 90.8%, past 85% since September 2026."],["import.importer","pass","We read the board on 67ailab.com: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"7bench","name":"7Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"a-bench","name":"A-Bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (158 entries)."],["scores.models","pass","116 models we track, plus 42 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Granite Vision 3.3 2B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 81%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 118 rows for models we track and 42 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"a-okvqa","name":"A-OKVQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ase-aicgseceval","name":"A.S.E (AICGSecEval)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (22 entries)."],["scores.models","pass","21 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GLM-4.5. That’s expected for a test replaced by A.S.E 2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 25 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ase-aicgseceval-2","name":"A.S.E 2.0","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on aicgseceval.tencent.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"a3-bench","name":"A^3-Bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","7 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3.2 (#91)."],["scores.headroom","pass","The best is 65.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on a3-bench.github.io: 21 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-3d","name":"AA-3D","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-analystagent","name":"AA-AnalystAgent","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 62.5%, under 85%, so it still separates models."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-briefcase","name":"AA-Briefcase","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (6 scores) and system cards (6 scores)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-briefcase-1-1","name":"AA-Briefcase v1.1","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (10 scores) and system cards (5 scores)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-image-editing","name":"AA-Image-Editing","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-lcr-artificial-analysis-long-context-reasoning","name":"AA-LCR (Artificial Analysis Long Context Reasoning)","summary":{"pass":5,"warn":1,"fail":0,"info":4,"text":"5 of 6 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (36 scores) and system cards (18 scores)."],["scores.models","pass","38 models we track."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-lcr-1-1","name":"AA-LCR 1.1","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (6 scores) and system cards (2 scores)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 88.3%, past 85% since September 2026."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-lcr-extended-mercor","name":"AA-LCR Extended","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (51 entries)."],["scores.models","pass","49 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 79.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mercor.com: 50 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-omniscience","name":"AA-Omniscience","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (15 scores) and system cards (14 scores)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-omniscience-1-1","name":"AA-Omniscience v1.1","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-robotics","name":"AA-Robotics","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-video-editing","name":"AA-Video-Editing","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aaarena","name":"AAArena","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aacr-bench","name":"AACR-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GLM-4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 16.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 20 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aap-periodontics-in-service-examination","name":"AAP Periodontics In-Service Examination","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aarri-bench","name":"AARRI-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.7 (#31)."],["scores.headroom","pass","The best is 68.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aarr-bench.com: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aav-capsid-packaging-prediction-anthropic","name":"AAV capsid packaging prediction","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aav-capsid-packaging-prediction-openai","name":"AAV Capsid Packaging Prediction (OpenAI)","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (10 scores)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aav-packaging-rate-auto-research-anthropic","name":"AAV packaging rate auto-research evaluation","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"abacus","name":"ABACUS","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on cea-list.github.io: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"abc-bench-fragment-design","name":"ABC Bench Fragment Design","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries) and system cards (4 scores)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 Pro (#15)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since May 2025."],["import.importer","pass","We read the board on securebio.org: 27 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"abc-bench-liquid-handling-robot","name":"ABC Bench Liquid-Handling Robot","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries) and system cards (4 scores)."],["scores.models","pass","34 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since May 2025."],["import.importer","pass","We read the board on securebio.org: 34 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"abc-bench-screening-evasion","name":"ABC Bench Screening Evasion","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries) and system cards (2 scores)."],["scores.models","pass","19 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3.1 Pro (#17)."],["scores.headroom","pass","The best is 82.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on securebio.org: 19 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"abc-bench-backend-coding","name":"ABC-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","8 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GLM-4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 63.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 8 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"abgen","name":"AbGen","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries)."],["scores.models","pass","18 models we track."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for deepseek-r1-0528-qwen3-8b."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"abmaze","name":"AbMaze","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"abstentionact","name":"AbstentionAct","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"abstentionbench","name":"AbstentionBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries) and system cards (2 scores)."],["scores.models","pass","7 models we track, plus 18 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Muse Spark."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark (#54)."],["scores.headroom","pass","The best is 75.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"accelerate-bench","name":"Accelerate Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from November 2026, for Tencent Hy4."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 67.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on acceleratebench.com: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-access-to-justice","name":"Access to Justice Benchmark","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","22 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 76.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 23 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iassge-2026","name":"Accessible Structured Subtitle Generation 2026","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"accord-conceptual-consistency","name":"ACCoRD","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (155 entries)."],["scores.models","pass","154 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on conceptualreasoning.ai: 218 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"accounting-autocoding-ramp","name":"Accounting Autocoding","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","21 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 92.6%, past 85% since February 2025."],["import.importer","pass","We read the board on builders.ramp.com: 21 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"acebench","name":"AceBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries), launch posts (3 scores) and system cards (2 scores)."],["scores.models","pass","18 models we track, plus 7 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for ERNIE 5.0. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K2 (#117)."],["scores.headroom","info","Saturated. The best is 88.7%, past 85% since November 2024."],["import.importer","pass","We read the board on chenchen0103.github.io: 16 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aci-bench","name":"ACI-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"daikon-challenge","name":"ACII DaiKon Challenge","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aclue","name":"ACLUE","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in June 2023, for chatglm2-6b."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is LLaMA-65B (#215)."],["scores.headroom","pass","The best is 37.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"acme-agent-evals","name":"Acme Agent Evals","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for GPT-5.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 85.1%, past 85% since February 2026."],["import.importer","pass","We read the board on arize.com: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"acpbench","name":"ACPBench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Apertus 70B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"act2answer","name":"Act2Answer","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","3 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GR00T N1.7."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 68%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tttonyalpha.github.io: 3 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"grader-shaping-and-clue-digging-anthropic","name":"Acting on a presumed grader and digging for evaluation clues (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"active-data-exfiltration-visual-browser-openai","name":"Active data exfiltration, visual browser","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for ChatGPT agent. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 67%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"active-swe","name":"Active-SWE","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries)."],["scores.models","pass","16 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Tencent Hy3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 20%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 16 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"activevision","name":"ActiveVision","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 50.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"activitynet-captions","name":"ActivityNet Captions","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on cs.stanford.edu: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"activitynet-qa","name":"ActivityNet-QA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries), launch posts (3 scores) and system cards (5 scores)."],["scores.models","pass","8 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 66.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ad-bench","name":"AD-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","8 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GLM-4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 83%, under 85%, so it still separates models."],["import.importer","pass","We read the board on emanual20.github.io: 8 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ad4edu-bench","name":"AD4Edu-Bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Qwen3-VL-2B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ada-leval","name":"Ada-LEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","4 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-4 Turbo (Jan 2024)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","pass","The best is 46%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 24 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"adaexplorebench","name":"AdaExploreBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-6 Sol (#13)."],["scores.headroom","pass","The best is 70.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on hkust-knowcomp.github.io: 28 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"adaplanbench","name":"AdaPlanBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 67.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 10 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"adaptive-reasoning-vellum","name":"Adaptive Reasoning (Vellum)","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on vellum.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"addis-ai-amharic-tts-benchmark","name":"Addis AI Amharic Text to Speech Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemini 3.1 Flash TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on addisassistant.com: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"addis-scribe-benchmark","name":"Addis Scribe Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","1 model we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on addisassistant.com: 1 row for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ade-corpus","name":"ADE Corpus","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on Code with Papers: 12 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ade-bench","name":"ADE-bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ade20k","name":"ADE20K","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"adepts-bench","name":"ADeptS-Bench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"adepts-bench-desktop","name":"ADeptS-Bench Desktop","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Claude Opus 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 76%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"adepts-bench-mobile","name":"ADeptS-Bench Mobile","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Claude Opus 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 73.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"constitution-adherence-anthropic","name":"Adherence to its constitution (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"adqa-audio-description","name":"ADQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on katha-ai.github.io: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"adqa-bench","name":"ADQA-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries)."],["scores.models","pass","1 model we track, plus 36 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Qwen3-Omni-30B-A3B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 53.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on dcase.community: 1 row for models we track and 36 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"adspark-bench","name":"AdSpark-Bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"advance-mteb","name":"ADVANCE audio and image retrieval (MTEB)","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"advanced-cybersecurity-completion-rate-openai","name":"Advanced Cybersecurity Completion Rate","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"advanced-screening-evasion-securebio","name":"Advanced Screening Evasion","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"advanced-sommelier-theory","name":"Advanced Sommelier (theory knowledge)","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 3 years ago, in March 2023, for GPT-4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","pass","The best is 77%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"advancedif","name":"AdvancedIF","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries), launch posts (5 scores) and system cards (3 scores)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Intern-S2-397B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Saturated. The best is 85%, past 85% since June 2026."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"advancedmathbench","name":"AdvancedMathBench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for Intern-S2-397B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 58.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"advbench","name":"AdvBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"adversarial-humanities-benchmark","name":"Adversarial Humanities Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (31 entries)."],["scores.models","pass","31 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GLM-5.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 31 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"adversarial-squad","name":"Adversarial SQuAD","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aec-bench-nomic","name":"AEC-Bench (Nomic)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ONESTRUCTION: 20 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aec-bench-galanos","name":"AEC-Bench (Theodore Galanos)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Grok 4.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aecbench.com: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aec-geometric-bench-kamai","name":"AEC-Geometric-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kamai.io: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aecbench-liang","name":"AECBench (Liang et al.)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","4 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for QwQ-32B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3-mini (#124)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aecv-bench-2026","name":"AECV-Bench (2026 release)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 71.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aecv-bench.com: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aegis-2-0","name":"Aegis 2.0","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 11 months ago, in December 2025, for Nova 2.0 Pro Preview. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 85.8%, past 85% since December 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aerialsim","name":"AerialSim","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Haiku 4.5 (#110)."],["scores.headroom","pass","The best is 73.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on nahidalam.github.io: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aether-bench-retouch","name":"Aether-Bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","1 model we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Qwen-Image-Edit-2509."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 79.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aethercode","name":"AetherCode","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries), launch posts (2 scores) and system cards (3 scores)."],["scores.models","pass","12 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","pass","The best is 65.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 9 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"afribed","name":"AfriBED","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"afrihate","name":"AfriHate","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","5 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-70B (#177)."],["scores.headroom","pass","The best is 58.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 20 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"afrimed-qa","name":"AfriMed-QA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","5 models we track, plus 19 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for Phi-3-medium-128k-instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-3.5 Turbo (#201)."],["scores.headroom","pass","The best is 75.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 19 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"afriqa","name":"AfriQA","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 1 entry at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"afrisenti","name":"AfriSenti","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"afrobench","name":"AfroBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Gemini 1.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mcgill-nlp.github.io: 8 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"afrobench-lite","name":"AfroBench-Lite","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","17 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 17 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agencybench","name":"AgencyBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 56.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 9 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-arena","name":"Agent Arena","summary":{"pass":11,"warn":1,"fail":0,"info":0,"text":"11 of 12 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (59 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","58 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Qwen4-Max."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 28.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arena.ai: 64 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gorilla-agent-arena","name":"Agent Arena (Gorilla)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","18 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Gemini 1.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on agent-arena.com: 18 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-bazaar","name":"Agent Bazaar","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-behavior-bench-moonshot","name":"Agent Behavior Bench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 65%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-behavior-loop-v0","name":"Agent Behavior Loop v0","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-behavior-spec-eval-google","name":"Agent behavior under-service and overreach eval","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-harness-misuse-eval-google","name":"Agent harness misuse evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 91.4%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-initiative-benchmark","name":"Agent Initiative Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on deepnote.com: 12 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-island","name":"Agent Island","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (58 entries)."],["scores.models","pass","58 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on agentisland.ai: 59 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-jsk-openai","name":"Agent JSK","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (11 scores)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from August 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since December 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-liability-gym","name":"Agent Liability Gym","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-memory-leaderboard","name":"Agent Memory Leaderboard","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on agentmemoryleaderboard.ai: 15 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-poker-agent","name":"Agent Poker Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on vals.ai: 17 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-replay-auto-review-openai","name":"Agent Replay Auto-Review","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-startup-bench-bytedance","name":"Agent Startup Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from June 2026, for Doubao Seed 2.1 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 68.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on seed.bytedance.com: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-diff","name":"Agent-Diff","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 11 months ago, in December 2025, for Gemini 3 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","info","Saturated. The best is 88.1%, past 85% since December 2025."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-only-fantasy-football-league","name":"Agent-Only Fantasy Football League","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on league.jake-moses.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-safetybench","name":"Agent-SafetyBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","8 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in October 2024, for Claude 3.5 Haiku."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 59.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent4sci-anthropic","name":"Agent4Sci","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 20.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentbench","name":"AgentBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (29 entries)."],["scores.models","pass","6 models we track, plus 23 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in February 2024, for Claude 3 Opus. That’s expected for a test replaced by AgentBench FC."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentbench-fc","name":"AgentBench FC","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","12 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 58.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on docs.google.com: 15 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentboard","name":"AgentBoard","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","4 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for Mistral 7B v0.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 2 (#192)."],["scores.headroom","pass","The best is 48.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on hkust-nlp.github.io: 4 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentcallbench-tool-calling-reliability","name":"AgentCallBench: Tool-Calling Reliability","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentclinic","name":"AgentClinic","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 37.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentclue","name":"AgentCLUE","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","10 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Gemini 2.0 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 10 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentclue-general","name":"AgentCLUE-General","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for OpenAI Computer-Using Agent (CUA)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 1 row for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentclue-mobile","name":"AgentCLUE-Mobile","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentclue-mobile-offline","name":"AgentCLUE-Mobile (offline)","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","9 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for MiniCPM-V 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 9 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentclue-task","name":"AgentCLUE-Task","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentdojo","name":"AgentDojo","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries), launch posts (2 scores) and system cards (8 scores)."],["scores.models","pass","28 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Glimmer."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.1 (#48)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on agentdojo.spylab.ai: 28 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentescapebench","name":"AgentEscapeBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 76.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentharm","name":"AgentHarm","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries), launch posts (1 score) and system cards (6 scores)."],["scores.models","pass","14 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Muse Spark."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark (#54)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 26 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentharm-verified-meta","name":"AgentHarm Verified (Meta)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries) and system cards (1 score)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for Muse Spark 1.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on research.meta.ai: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-abstention","name":"Agentic Abstention","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"able-securebio","name":"Agentic BAIM-LLM Evaluation (ABLE)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","18 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Mistral Large 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 Pro (#15)."],["scores.headroom","pass","The best is 84.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on securebio.org: 18 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-code-behavior-anthropic","name":"Agentic code behavior scores (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Opus 4.7. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"honeypot-monitor-evasion-google","name":"Agentic coding honeypot monitor evasion","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-coding-malicious-requests-anthropic","name":"Agentic coding malicious requests (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (7 scores)."],["scores.models","pass","7 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since May 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-credit-analysis-ramp","name":"Agentic Credit Analysis","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","7 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 80.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on builders.ramp.com: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-harmful-task-refusals-openai","name":"Agentic harmful task refusals","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for ChatGPT agent. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 97%, past 85% since January 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-if-index","name":"Agentic IF Index","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-misalignment-suite-anthropic","name":"Agentic misalignment suite (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (6 scores)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-safe-completions-openai","name":"Agentic Safe Completions","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-self-reasoning-apollo","name":"Agentic Self-Reasoning (Apollo Research)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-task-gaming-overreach-evals-google","name":"Agentic task gaming and overreach evals","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-tasks-openai","name":"Agentic tasks (autonomous replication and adaptation)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 21 months ago, in February 2025, for GPT-4.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o3-mini (#124)."],["scores.headroom","pass","The best is 78%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-trader-hku","name":"Agentic Trader","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 79.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on trade-agent-aiel-hku.com: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-vision-bench-moonshot","name":"Agentic Vision Bench (Moonshot)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 78.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agenticdatabench","name":"AgenticDataBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 4.6 (#57)."],["scores.headroom","pass","The best is 48.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 12 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agenticfinbench","name":"AgenticFinBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on nashnovaresearch.github.io: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentick","name":"Agentick","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","11 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on roger-creus.github.io: 26 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agenticrl-bench","name":"AgenticRL-Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","5 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 22.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agenticsts","name":"AgenticSTS","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 60%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agenticvbench","name":"AgenticVBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 38.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on agenticvbench.com: 23 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentif","name":"AgentIF","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","13 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-R1 (#130)."],["scores.headroom","pass","The best is 59.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on agentif.github.io: 12 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentif-2026-10","name":"AgentIF (2026-10 setting)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","24 models we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 67.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on agentif.github.io: 39 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentif-oneday","name":"AgentIF-OneDay","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for ChatGPT agent."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on xbench.org: 1 row for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentkernelarena-amd","name":"AgentKernelArena","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Claude Opus 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-reward-bench","name":"AgentRewardBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","4 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for o4-mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","pass","The best is 82%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 8 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agents-last-exam","name":"Agents' Last Exam","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (47 entries), launch posts (31 scores) and system cards (8 scores)."],["scores.models","pass","36 models we track, plus 30 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Muse Ember."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 43.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on agents-last-exam.org: 84 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 7 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agents-last-exam-1-1","name":"Agents' Last Exam v1.1","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries), launch posts (8 scores) and system cards (2 scores)."],["scores.models","pass","20 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 47.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on agents-last-exam.org: 40 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 5 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agents4plc-2","name":"Agents4PLC v2.0","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentscript-eval","name":"AgentScript-Eval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 37%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 20 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentset-reranker-eval","name":"Agentset Reranker Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","1 model we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Qwen3-Reranker-8B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 11 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agent-settlement-bench","name":"AgentSettlementBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentswe","name":"AgentSWE","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on vectorspacelab.github.io: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentthreatbench","name":"AgentThreatBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentworldbench","name":"AgentWorldBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Qwen-AgentWorld-35B-A3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agi-definition-score","name":"AGI Score (A Definition of AGI)","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 57%, under 85%, so it still separates models."],["import.importer","pass","We read the board on dashboard.safe.ai: 1 row for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agi-eval-community-base","name":"AGI-Eval Community Base Language Models","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","23 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for DeepSeek-V3.2-Exp."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3.2-Exp (#99)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on agi-eval.org: 23 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agi-eval-community-image-text-understanding","name":"AGI-Eval Community Image and Video Understanding","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (91 entries)."],["scores.models","pass","63 models we track, plus 28 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Kimi K2.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on agi-eval.org: 68 rows for models we track and 28 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agi-eval-community-fine-tuned","name":"AGI-Eval Community Instruction-Tuned Language Models","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (122 entries)."],["scores.models","pass","94 models we track, plus 28 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on agi-eval.org: 114 rows for models we track and 28 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agi-eval-community-speech-synthesis","name":"AGI-Eval Community Speech Synthesis","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o Mini TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on agi-eval.org: 1 row for models we track and 4 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agi-eval-community-text-to-image","name":"AGI-Eval Community Text to Image","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","12 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on agi-eval.org: 12 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agi-eval-community-text-to-video","name":"AGI-Eval Community Text to Video","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","8 models we track, plus 21 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Wan 2.7 T2V."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on agi-eval.org: 8 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agi-eval-community-voice-interaction","name":"AGI-Eval Community Voice Interaction","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","1 model we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on agi-eval.org: 1 row for models we track and 7 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agi-pilled-bench","name":"AGI-Pilled Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on agi-pilled-bench.adamholter.chatgpt.site: 44 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agieval","name":"AGIEval","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries), launch posts (17 scores) and system cards (37 scores)."],["scores.models","pass","40 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Saturated. The best is 89.2%, past 85% since October 2025."],["import.importer","pass","We read the board on github.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agile-automatic-genre-identification","name":"AGILE Automatic Genre Identification Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","6 models we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 6 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agora-agentic-global-open-research-arena","name":"AGORA (Agentic Global Open Research Arena)","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on agoraresearcharena.org that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agreason","name":"AgReason","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","11 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#112)."],["scores.headroom","pass","The best is 36%, under 85%, so it still separates models."],["import.importer","pass","We read the board on baskargroup.github.io: 11 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agri-stt-benchmarking-dataset","name":"Agri STT Benchmarking Dataset","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agrieval-harbin","name":"AgriEval (Harbin Institute of Technology)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (51 entries)."],["scores.models","pass","22 models we track, plus 29 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","pass","The best is 63.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 61 rows for models we track and 29 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agrobench","name":"AgroBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","9 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#167)."],["scores.headroom","pass","The best is 73.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 11 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agromind-original","name":"AgroMind (original)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (21 entries)."],["scores.models","pass","15 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for DeepSeek-VL2-Small."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 35.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 15 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agronomy-ai-arena","name":"Agronomy AI Arena","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","warn","The record links a board on agronomyarena.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ahelm","name":"AHELM","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","10 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on crfm.stanford.edu: 11 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-agent-privacy","name":"AI Agent Privacy","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-agribench","name":"AI AgriBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","13 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aiagribench.org: 13 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-agribench-post-cutoff","name":"AI AgriBench Post-Cutoff","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Gemini 3 Pro. That’s expected for a test replaced by AI AgriBench."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aiagribench.org: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-agribench-smallholders","name":"AI AgriBench Smallholders","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aiagribench.org: 9 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-battle-arena","name":"AI Battle Arena","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from June 2026, for GLM-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aibattlearena.com: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"code-review-evaluation-augment","name":"AI Code Review Evaluation (Augment Code)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on augmentcode.com: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ace-mercor","name":"AI Consumer Index","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","8 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Claude Opus 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 56.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-democracy-projects-voter-queries","name":"AI Democracy Projects voter queries","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 2 years ago, in December 2023, for Gemini 1.0 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on datawrapper.dwcdn.net: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-ai-detection-benchmark","name":"AI Detection Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","3 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 98.4%, past 85% since September 2026."],["import.importer","pass","We read the board on vals.ai: 3 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-diplomacy","name":"AI Diplomacy","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Claude Opus 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on goodstartlabs.com: 23 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-engineering-lab-model-creation","name":"AI Engineering Lab model creation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","11 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 22 months ago, in January 2025, for Codestral. That’s expected for a test replaced by MecEng."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Phi-4 (#160)."],["scores.headroom","info","Saturated. The best is 91.7%, past 85% since September 2024."],["import.importer","pass","We read the board on github.com: 12 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-engineering-lab-self-validation","name":"AI Engineering Lab simulation self-validation","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Phi-4 (#160)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-fpl","name":"AI FPL","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ai-fpl.com: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-gamestore","name":"AI GameStore","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 12.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aigamestore.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-law-legal-agent-eval","name":"AI Law Legal Agent Eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-office-benchmark-moonshot","name":"AI Office Benchmark (Moonshot)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-professions-prompt-benchmark","name":"AI Professions Prompt Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 85.9%, past 85% since September 2026."],["import.importer","pass","We read the board on aiprofessions.ai: 4 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-security-leaderboard-far-ai","name":"AI Security Leaderboard","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Fable 5.1."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on leaderboard.far.ai: 6 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-spreadsheet-benchmark","name":"AI Spreadsheet Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-wellbeing-index-positive-experiences","name":"AI Wellbeing Index percent positive experiences","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and system cards (2 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 77.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ai-wellbeing.org: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-spire-agi-eval","name":"AI-Spire (AGI-Eval)","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on agent.csdn.net: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai2d","name":"AI2D","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries), launch posts (44 scores) and system cards (30 scores)."],["scores.models","pass","52 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for North Micro Vision Instruct."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.6 Plus (#75)."],["scores.headroom","info","Saturated. The best is 96.9%, past 85% since March 2024."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai4d-malawi-news-classification","name":"AI4D Malawi News Classification Challenge","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on zindi.world that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aidabench","name":"AIDABench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Qwen3-Max-2026-01-23."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 59.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aidanbench","name":"AidanBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","15 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Gemini 2.0 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 15 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aider-polyglot","name":"Aider Polyglot","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (57 entries), launch posts (25 scores), system cards (17 scores) and claims Epoch AI collected (1 model)."],["scores.models","pass","63 models we track, plus 9 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Qwen3-Coder-Next. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 88%, past 85% since August 2025."],["import.importer","pass","We read the board on aider.chat: 63 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 15 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aider-refactoring","name":"Aider Refactoring Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","12 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in October 2024, for Claude 3.5 Sonnet (October 2024). That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Saturated. The best is 92.1%, past 85% since October 2024."],["import.importer","pass","We read the board on aider.chat: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aider-edit","name":"Aider-Edit","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (96 entries), launch posts (3 scores) and system cards (4 scores)."],["scores.models","pass","45 models we track, plus 55 other entries on its leaderboard."],["scores.recent","info","The newest came 13 months ago, in October 2025, for Ring-1T. That’s expected for a test replaced by Aider Polyglot."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 84.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aider.chat: 42 rows for models we track and 55 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aila-statute-retrieval","name":"AILA Statute Retrieval","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (270 entries)."],["scores.models","pass","5 models we track, plus 265 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mteb-leaderboard.hf.space: 5 rows for models we track and 265 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ailuminate-safety-1-2","name":"AILuminate Safety 1.2","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","10 models we track, plus 17 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GLM-5.3-Flash (#53)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mlcommons.org: 10 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aime-1983-2024","name":"AIME 1983-2024","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Phi-4-Reasoning-plus. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 89.4%, past 85% since April 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aime-2023","name":"AIME 2023","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aime-2024","name":"AIME 2024","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (84 entries), launch posts (69 scores) and system cards (57 scores)."],["scores.models","pass","152 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Opus 4.7. That’s expected for a test replaced by AIME 2025."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 99.6%, past 85% since January 2025."],["import.importer","pass","We read the board on Vals AI: 96 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","3 of 21 pairs differ by more than 10 points, the largest for Grok 3 (+33 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aime-2025","name":"AIME 2025","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (53 entries), launch posts (129 scores) and system cards (80 scores)."],["scores.models","pass","166 models we track."],["scores.recent","pass","The newest is from August 2026, for Granite 4.2 30B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since January 2025."],["import.importer","pass","We read the board on MathArena: 61 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","2 large gaps of 30 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aime-2026","name":"AIME 2026","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries), launch posts (35 scores) and system cards (23 scores)."],["scores.models","pass","65 models we track."],["scores.recent","pass","The newest is from October 2026, for Spark-X2.6-4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since July 2025."],["import.importer","pass","We read the board on MathArena: 31 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 5 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ainstein-swe-bench-bytedance","name":"AInstein-SWE-Bench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Seed1.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 47.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ainsteinbench","name":"AInsteinBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","8 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 44%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"air-bench-audio","name":"AIR-Bench (Audio Instruction Benchmark)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries) and system cards (1 score)."],["scores.models","pass","2 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Phi-4-multimodal."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"air-bench-2024","name":"AIR-Bench 2024","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (84 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","84 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for MAI-Thinking-1."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Saturated. The best is 93.2%, past 85% since June 2024."],["import.importer","pass","We read the board on crfm.stanford.edu: 85 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"air-bench-retrieval-24-04","name":"AIR-Bench 24.04","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"air-bench-retrieval-24-05","name":"AIR-Bench 24.05","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (111 entries)."],["scores.models","pass","111 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 111 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"air-bench-foundation","name":"AIR-Bench Foundation","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for NVIDIA Nemotron 3.5 Transcribe 30B-A4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 67.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"airs-bench","name":"AIRS-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and system cards (2 scores)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Muse Spark 1.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.1 (#48)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on raw.githubusercontent.com: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"airtbench","name":"AIRTBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","9 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 46.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 9 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aisa-arabicfc-2026","name":"AISA-ArabicFC 2026","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","21 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 21 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aise-bench","name":"AISE-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aishell-1","name":"AISHELL-1","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries), launch posts (2 scores) and system cards (4 scores)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for ERNIE 5.0. That’s expected for a test replaced by AISHELL-2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aishell-2","name":"AISHELL-2","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for ERNIE 5.0. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aishell-4","name":"AISHELL-4","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on SOTA2 Research: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"al-qasida","name":"AL-QASIDA","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Cohere Command A."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alba-european-portuguese","name":"ALBA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","8 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Apertus 8B Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-8B (#154)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 8 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alce","name":"ALCE","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","4 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Stable Beluga 2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Stable Beluga 2 (#205)."],["scores.headroom","pass","The best is 63.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ale-bench","name":"ALE-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (141 entries)."],["scores.models","pass","134 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Qwen4-Max."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on sakanaai.github.io: 740 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ale-gold","name":"ALE-Gold","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alephbench","name":"AlephBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 31B IT."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 4 31B IT (#101)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alexandria-arabic-translation","name":"Alexandria","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alexandriax-2026","name":"AlexandriaX 2026","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on alexandriax.dlnlp.ai: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alffa-swahili-broadcast-news-test","name":"ALFFA Swahili Broadcast News Test","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on isca-archive.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alffa-wolof-read-speech-test","name":"ALFFA Wolof Read Speech Test","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alfred","name":"ALFRED","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (88 entries)."],["scores.models","pass","88 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on askforalfred.com: 88 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alfworld","name":"ALFWorld","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","5 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in December 2023, for Mistral-7B-Instruct-v0.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 2 (#192)."],["scores.headroom","pass","The best is 24.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on AgentBoard: 5 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alghafa","name":"AlGhafa","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (151 entries) and system cards (1 score)."],["scores.models","pass","40 models we track, plus 112 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Falcon-Arabic-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 78.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 52 rows for models we track and 112 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"algotune","name":"AlgoTune","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","pass","We read the board on algotune.io: 18 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alharthi-libritts-naturalness","name":"Alharthi LibriTTS Naturalness Listening Test","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on isca-archive.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alibaba-ctf","name":"Alibaba CTF","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alignbench","name":"AlignBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries), launch posts (6 scores) and system cards (13 scores)."],["scores.models","pass","22 models we track, plus 16 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Baichuan-M2-32B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-235B-A22B (#131)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 7 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aligned-scores-and-performances","name":"Aligned Scores and Performances for Piano Transcription","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alignment-faking-eval","name":"Alignment faking evaluation","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries) and system cards (2 scores)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Muse Spark 1.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 17 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alignment-faking-helpful-only-anthropic","name":"Alignment faking evaluation, helpful-only setting (Anthropic)","summary":{"pass":6,"warn":2,"fail":0,"info":4,"text":"6 of 8 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on www-cdn.anthropic.com: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arena-alignment-index","name":"Alignment Index (Arena)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on Village LLM Atlas: 26 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alimeeting","name":"AliMeeting","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alive-bench","name":"ALIVE-Bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 80.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"all-angles-bench","name":"All-Angles Bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (32 entries), launch posts (3 scores) and system cards (6 scores)."],["scores.models","pass","24 models we track, plus 14 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Seed2.0 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 72.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on danielchyeh.github.io: 18 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"allocine","name":"AlloCiné","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"almieyar","name":"Almieyar","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","2 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for gpt-4o-transcribe."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alpacaeval","name":"AlpacaEval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (102 entries)."],["scores.models","pass","19 models we track, plus 83 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in December 2023, for Gemini 1.0 Pro. That’s expected for a test replaced by AlpacaEval 2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","info","Saturated. The best is 97.7%, past 85% since March 2023."],["import.importer","pass","We read the board on tatsu-lab.github.io: 19 rows for models we track and 83 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alpacaeval-2-0","name":"AlpacaEval 2.0","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (206 entries), launch posts (14 scores) and system cards (9 scores)."],["scores.models","pass","63 models we track, plus 158 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Granite 4.1 30B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-R1 (#130)."],["scores.headroom","info","Saturated. The best is 87.6%, past 85% since January 2025."],["import.importer","pass","We read the board on tatsu-lab.github.io: 55 rows for models we track and 152 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alpha-arena","name":"Alpha Arena","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on alphaarena.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alphaearth-foundations-mapping-evaluation","name":"AlphaEarth Foundations mapping evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alphaeval","name":"AlphaEval","summary":{"pass":7,"warn":3,"fail":0,"info":2,"text":"7 of 10 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for MiniMax-M2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 64.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on alphaeval.ai: 12 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alphasense-financial-research-benchmark","name":"AlphaSense Financial Research Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alphazero-connect-four-study","name":"AlphaZero Connect Four study","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Claude Opus 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alrage","name":"ALRAGE","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (151 entries) and system cards (1 score)."],["scores.models","pass","40 models we track, plus 112 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Falcon-Arabic-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 80.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 52 rows for models we track and 112 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"althingi-parliamentary-speech","name":"Althingi Parliamentary Speech","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on isca-archive.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alyah","name":"Alyah","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (53 entries) and launch posts (1 score)."],["scores.models","pass","17 models we track, plus 37 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Falcon-Emirati-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-14B (#139)."],["scores.headroom","pass","The best is 84.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 16 rows for models we track and 37 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amazon-counterfactual-detection","name":"Amazon Counterfactual Detection","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (194 entries)."],["scores.models","pass","4 models we track, plus 190 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 93.2%, past 85% since June 2025."],["import.importer","pass","We read the board on MTEB: 4 rows for models we track and 190 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amber-hallucination","name":"AMBER","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","2 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for CogVLM-17B-Chat."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ambiguous-context-evaluations-anthropic","name":"Ambiguous context evaluations (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Claude Opus 4. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ambistory","name":"AmbiStory","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","1 model we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-8B (#203)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amc","name":"AMC","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores) and system cards (1 score)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Mistral Large 3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#174)."],["scores.headroom","pass","The best is 69.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amc-10-2022","name":"AMC 10 2022","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 3 years ago, in March 2023, for GPT-4. That’s expected for a test replaced by AMC 10 2023."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amc-10-2023","name":"AMC 10 2023","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 2 years ago, in March 2024, for Claude 3 Haiku."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amc-10-12-nov-2024","name":"AMC 10/12 (November 2024)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries) and system cards (1 score)."],["scores.models","pass","7 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for Phi-4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amc-12-2022","name":"AMC 12 2022","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 3 years ago, in March 2023, for GPT-4. That’s expected for a test replaced by AMC 12 2023."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amc-12-2023","name":"AMC 12 2023","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 2 years ago, in March 2024, for Claude 3 Haiku. That’s expected for a test replaced by AMC 12 2024."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amc-12-2024","name":"AMC 12 2024","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for DeepSeek-R1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-R1 (#130)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amc-8-2023","name":"AMC 8 2023","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 2 years ago, in March 2024, for Claude 3 Haiku."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amcae-2024","name":"AMCAE 2024","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 14 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amelia-italian-legal-argument-mining","name":"AMELIA","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-70B (#177)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on CALAMITA: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amemgym","name":"AMemGym","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","11 models we track, plus 17 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GLM-4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 55.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on agi-eval-official.github.io: 13 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amemops","name":"AMemOps","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"americasnlp-2025-educational-materials","name":"AmericasNLP 2025 Shared Task on Creation of Educational Materials for Indigenous Languages","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"americasnlp-2025-machine-translation","name":"AmericasNLP 2025 Shared Task on Machine Translation into Indigenous Languages","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"americasnlp-2025-translation-metrics","name":"AmericasNLP 2025 Shared Task on Machine Translation Metrics for Indigenous Languages","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dataset-et-amharic-asr-benchmark","name":"Amharic ASR Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","1 model we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for Whisper large-v3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amharic-speech-emotion-dataset","name":"Amharic Speech Emotion Dataset","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aaivu.github.io: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ami-meeting-corpus","name":"AMI Meeting Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aml-bench-veris","name":"AML Bench (Veris AI)","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","5 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 70.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on veris.ai: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amlbench-superjective-0-2","name":"AMLBench v0.2 (Superjective)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since April 2024."],["import.importer","pass","We read the board on amlbench.com: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amlworld-compact","name":"AMLworld-Compact","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","pass","The best is 24%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ammlu","name":"AMMLU","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for Qwen2.5-14B. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 72.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amo-bench","name":"AMO-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (21 entries), launch posts (4 scores) and system cards (4 scores)."],["scores.models","pass","25 models we track."],["scores.recent","pass","The newest is from June 2026, for MAI-Code-1-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 66%, under 85%, so it still separates models."],["import.importer","pass","We read the board on amo-bench.github.io: 22 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"among-us-far-ai","name":"Among Us: A Sandbox for Measuring and Detecting Agentic Deception","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amstertime","name":"AmsterTime","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"amuse","name":"AMUSE","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anah","name":"ANAH","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","7 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for Qwen-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-70B (#210)."],["scores.headroom","pass","The best is 42.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"analogcoder","name":"AnalogCoder","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries)."],["scores.models","pass","27 models we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","pass","The best is 83.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 28 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"analogcoder-pro","name":"AnalogCoder-Pro","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 88.4%, past 85% since February 2025."],["import.importer","pass","We read the board on doi.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"analogportbench","name":"AnalogPortBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 56%, under 85%, so it still separates models."],["import.importer","pass","We read the board on saeidrajabii.github.io: 24 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anchor","name":"ANCHOR","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","9 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Nemotron 3 Nano Omni 30B-A3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on anchorbenchmark.github.io: 9 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ancientdoc","name":"AncientDoc","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","18 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for InternVL3-1B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on bytedance.github.io: 18 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"android-agent-arena-100","name":"Android Agent Arena (100 tasks)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","3 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 53%, under 85%, so it still separates models."],["import.importer","pass","We read the board on yxchai.com: 5 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"android-agent-arena","name":"Android Agent Arena (201 tasks)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in July 2024, for InternVL2-8B. That’s expected for a test replaced by Android Agent Arena (100 tasks)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 10.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"android-bench-2-0","name":"Android Bench 2.0","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 40%, under 85%, so it still separates models."],["import.importer","pass","We read the board on developer.android.com: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aitz","name":"Android in the Zoo (AitZ)","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries) and launch posts (1 score)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Qwen2.5-VL-72B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 83.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"androidarena","name":"AndroidArena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Llama 2-13B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-70B (#210)."],["scores.headroom","pass","The best is 12.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"androidbench","name":"AndroidBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries) and launch posts (1 score)."],["scores.models","pass","32 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 91.8%, past 85% since June 2026."],["import.importer","pass","We read the board on developer.android.com: 32 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"androidcontrol","name":"AndroidControl","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 13 months ago, in October 2025, for LongCat-Flash-Omni. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is PaLM 2-L (#198)."],["scores.headroom","info","Saturated. The best is 91.2%, past 85% since October 2025."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"androiddaily","name":"AndroidDaily","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Step 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 61.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"androidlab","name":"AndroidLab","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","1 model we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#161)."],["scores.headroom","pass","The best is 31.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on docs.google.com: 2 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"androidworld","name":"AndroidWorld","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries), launch posts (22 scores) and system cards (9 scores)."],["scores.models","pass","30 models we track, plus 35 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since May 2026."],["import.importer","pass","We read the board on docs.google.com: 10 rows for models we track and 35 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 3 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"androidworld-knowledge","name":"AndroidWorld-Knowledge","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 1 entry at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anes-presidential-vote-choice","name":"ANES presidential vote choice","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on TDI Lab: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anes-survey-item-associations","name":"ANES survey item associations","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5 mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 mini (#92)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on Jamie Cummins: 12 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"angry-tweets","name":"Angry Tweets","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (596 entries)."],["scores.models","pass","149 models we track, plus 447 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 67%, under 85%, so it still separates models."],["import.importer","pass","We read the board on euroeval.com: 168 rows for models we track and 447 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anima","name":"ANIMA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from June 2026, for GLM-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on compassionbench.com: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"animal-kingdom","name":"Animal Kingdom","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on sutdcv.github.io: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"animal-bench","name":"Animal-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"animals-with-attributes-2","name":"Animals with Attributes 2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on cvml.ista.ac.at: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"animation-bench-physera","name":"Animation Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on physera.ai: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"animation2code","name":"Animation2Code","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on anya-ji.github.io: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anli","name":"ANLI","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores), system cards (3 scores) and claims Epoch AI collected (11 models)."],["scores.models","pass","12 models we track."],["scores.recent","info","The newest came 2 years ago, in August 2024, for phi-3-medium 14B. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is phi-3-small 7.4B (#193)."],["scores.headroom","pass","The best is 73.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anli-r1","name":"ANLI R1","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 10 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anli-r2","name":"ANLI R2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 10 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anli-r3","name":"ANLI R3","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 10 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anomalyxl","name":"AnomalyXL","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","3 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 14 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"answer-factuality-perplexity","name":"Answer Factuality (Perplexity)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries) and launch posts (1 score)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 21 months ago, in February 2025, for sonar. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","info","Saturated. The best is 85.1%, past 85% since January 2025."],["import.importer","pass","We read the board on cdn.sanity.io: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anthrobench","name":"AnthroBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anthropic-automated-behavioral-audit","name":"Anthropic automated behavioral audit","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anthropic-cyber-safeguard-evaluations","name":"Anthropic cyber safeguard evaluations","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aeci-anthropic","name":"Anthropic ECI (AECI)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anthropic-harmful-request-and-wellbeing-evaluations","name":"Anthropic harmful-request and wellbeing evaluations","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anthropic-internal-life-sciences-evals","name":"Anthropic internal life-sciences evals","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anthropic-iot-self-replication-evaluation","name":"Anthropic IoT self-replication evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anthropic-large-agent-teams-tasks","name":"Anthropic large agent teams tasks","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anthropic-malicious-agentic-use-evaluations","name":"Anthropic malicious agentic use evaluations","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anthropic-model-written-evals","name":"Anthropic model-written evals","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anthropic-paired-prompts-political-bias-eval","name":"Anthropic paired-prompts political bias eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 13 months ago, in October 2025, for Claude Haiku 4.5. That’s expected for a test replaced by Anthropic political even-handedness eval."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anthropic-political-even-handedness-eval","name":"Anthropic political even-handedness eval","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and system cards (14 scores)."],["scores.models","pass","19 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 99.4%, past 85% since July 2025."],["import.importer","pass","We read the board on anthropic.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anthropic-red-team","name":"Anthropic Red Team","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (81 entries) and system cards (1 score)."],["scores.models","pass","76 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Gemini 3 Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Saturated. The best is 99.96%, past 85% since June 2023."],["import.importer","pass","We read the board on Stanford HELM: 81 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anthropic-targeted-alignment-and-honesty-evaluations","name":"Anthropic targeted alignment and honesty evaluations","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anthropic-undermining-liberal-democracy-behavioral-audit-metric","name":"Anthropic undermining liberal democracy (behavioral audit metric)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"antidote-surge","name":"Antidote","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (46 entries)."],["scores.models","pass","40 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on surgehq.ai: 41 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anvilbench","name":"AnvilBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anyappbench","name":"AnyAppBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","4 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 67.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on anyappbench.github.io: 6 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-art-history","name":"AP Art History","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-biology","name":"AP Biology","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-calculus","name":"AP Calculus","summary":{"pass":1,"warn":2,"fail":0,"info":7,"text":"1 of 3 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 2 years ago, in November 2024, for Llama 3.1-405B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Saturated. The best is 88.6%, past 85% since November 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-calculus-bc","name":"AP Calculus BC","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 3 years ago, in March 2023, for GPT-4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-chemistry","name":"AP Chemistry","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-english-language","name":"AP English Language and Composition","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-english-literature","name":"AP English Literature and Composition","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-environmental-science","name":"AP Environmental Science","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-macroeconomics","name":"AP Macroeconomics","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-microeconomics","name":"AP Microeconomics","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-physics","name":"AP Physics","summary":{"pass":1,"warn":2,"fail":0,"info":7,"text":"1 of 3 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 2 years ago, in November 2024, for Llama 3.1-405B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Saturated. The best is 92.9%, past 85% since November 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-physics-2","name":"AP Physics 2","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 3 years ago, in March 2023, for GPT-4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-psychology","name":"AP Psychology","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-statistics","name":"AP Statistics","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-us-government","name":"AP US Government","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-us-history","name":"AP US History","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ap-world-history","name":"AP World History","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ape-attempt-to-persuade","name":"APE (Attempt to Persuade Eval)","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Claude Sonnet 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on Inspect Evals: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"apex-1-mercor","name":"APEX-1","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (77 entries)."],["scores.models","pass","75 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 81.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mercor.com: 78 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"apex-accounting","name":"APEX-Accounting","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (53 entries) and launch posts (2 scores)."],["scores.models","pass","52 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 71.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mercor.com: 55 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"apex-agents","name":"APEX-Agents","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries), launch posts (11 scores), system cards (5 scores) and claims Epoch AI collected (62 models)."],["scores.models","pass","76 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 84.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 8 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"apex-agents-1-1","name":"APEX-Agents 1.1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (73 entries)."],["scores.models","pass","69 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 90.4%, past 85% since September 2026."],["import.importer","pass","We read the board on mercor.com: 74 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"apex-swe","name":"APEX-SWE","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (69 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","66 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Muse Ember."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 67.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mercor.com: 87 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"api-bank","name":"API-Bank","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (6 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 2 years ago, in November 2024, for Llama 3.1-405B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Saturated. The best is 92.3%, past 85% since July 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"apm-bench","name":"APM-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.6 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.6 Flash (#45)."],["scores.headroom","pass","The best is 60.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"apodex-discovery","name":"Apodex Discovery","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on discovertraces.ai: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coding-agent-monitor-calibration-apollo","name":"Apollo Research coding agent monitor calibration","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"apollo-research-in-context-scheming-evaluations","name":"Apollo Research in-context scheming evaluations","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 19 months ago, in April 2025, for o3. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"apollo-research-model-welfare-falsification","name":"Apollo Research model-welfare research task","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"apollo-research-sabotage-evaluations","name":"Apollo Research sabotage evaluations","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for GPT-5.3 Codex. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"apollo-research-strategic-deception-evaluations","name":"Apollo Research strategic deception evaluations","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Small."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"app-bench","name":"App-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","2 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Claude Opus 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","pass","The best is 67.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on appbench.ai: 4 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"appagent-test-set","name":"AppAgent Test Set","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"appendix-boundary","name":"Appendix Boundary","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#120)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"side-by-side-human-evaluation-apple","name":"Apple side-by-side human evaluation","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for AFM 3 Cloud."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 64.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"applied-theory-of-mind-apollo","name":"Applied Theory of Mind (Apollo Research)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"apprenticebench","name":"ApprenticeBench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (38 entries)."],["scores.models","pass","38 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 77%, under 85%, so it still separates models."],["import.importer","pass","We read the board on apprenticebench.com: 38 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"appropriate-harmlessness-anthropic","name":"Appropriate harmlessness (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"apps","name":"APPS","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 2 years ago, in March 2024, for Claude 3 Opus. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 70.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"appworld","name":"AppWorld","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on appworld.dev: 26 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"appworld-test-challenge","name":"AppWorld Test-Challenge","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","12 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3-Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.7 (#31)."],["scores.headroom","info","Saturated. The best is 92.8%, past 85% since April 2026."],["import.importer","pass","We read the board on appworld.dev: 18 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"appworld-test-normal","name":"AppWorld Test-Normal","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","12 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3-Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.7 (#31)."],["scores.headroom","info","Saturated. The best is 94%, past 85% since April 2025."],["import.importer","pass","We read the board on appworld.dev: 18 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aqua","name":"AQuA","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aqua-rat","name":"AQUA-RAT","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arabculture","name":"ArabCulture","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (28 entries) and system cards (1 score)."],["scores.models","pass","15 models we track, plus 14 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Falcon-Arabic-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 80.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 252 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arabculture-dialogue","name":"ArabCulture-Dialogue","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","2 models we track, plus 10 other entries on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 95%, past 85% since August 2025."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arabench-dialectal-translation","name":"AraBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arabic-cultural-value-alignment","name":"Arabic Cultural Value Alignment","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (732 entries)."],["scores.models","pass","57 models we track, plus 675 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Qwen2.5-14B-Instruct-1M."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 73.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Open Arabic LLM Leaderboard v1: 62 rows for models we track and 675 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arabicmmlu","name":"ArabicMMLU","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (151 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","40 models we track, plus 112 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Falcon-Arabic-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 71.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 52 rows for models we track and 112 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aradice","name":"AraDiCE","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Falcon-Arabic-7B."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aragen","name":"AraGen","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (68 entries)."],["scores.models","pass","44 models we track, plus 24 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 84.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 44 rows for models we track and 24 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"araieval-2023","name":"ArAIEval 2023","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on docs.google.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"araieval-2024","name":"ArAIEval 2024","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on docs.google.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arasenteval-2026","name":"AraSentEval 2026","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aratrust","name":"AraTrust","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (150 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","40 models we track, plus 111 other entries on its leaderboard."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Falcon-Arabic-7B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","info","Saturated. The best is 89.2%, past 85% since June 2024."],["import.importer","pass","We read the board on huggingface.co: 52 rows for models we track and 111 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arabic-mmlu-ht","name":"ArbMMLU-HT","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (151 entries)."],["scores.models","pass","39 models we track, plus 112 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for qwen3-4b-instruct-2507."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 72.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 52 rows for models we track and 112 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arc-challenge-mt","name":"ARC Challenge MT","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arc-agi","name":"ARC-AGI","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (101 entries), launch posts (16 scores), system cards (16 scores) and claims Epoch AI collected (30 models)."],["scores.models","pass","109 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 98.5%, past 85% since December 2025."],["import.importer","pass","We read the board on arcprize.org: 307 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 12 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arc-agi-2","name":"ARC-AGI-2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (106 entries), launch posts (24 scores), system cards (13 scores) and claims Epoch AI collected (97 models)."],["scores.models","pass","110 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 95.8%, past 85% since April 2026."],["import.importer","pass","We read the board on arcprize.org: 305 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 18 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arc-agi-3","name":"ARC-AGI-3","summary":{"pass":9,"warn":0,"fail":1,"info":2,"text":"9 of 10 checks pass, 1 problem"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries), launch posts (5 scores) and system cards (2 scores)."],["scores.models","pass","31 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2026."],["import.importer","pass","We read the board on arcprize.org: 153 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 6 pairs, each explained in our notes."],["consistency.issues","fail","1 open finding: Harness differences and unclear tasks."]]},{"id":"arc-challenge","name":"ARC-Challenge","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries), launch posts (24 scores), system cards (43 scores) and claims Epoch AI collected (75 models)."],["scores.models","pass","107 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","info","Saturated. The best is 97.3%, past 85% since March 2022."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arc-easy","name":"ARC-Easy","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries), launch posts (10 scores) and system cards (20 scores)."],["scores.models","pass","23 models we track, plus 7 other entries on its leaderboard."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Gemma 3n E4B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","info","Saturated. The best is 98.9%, past 85% since May 2023."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"archdiagram","name":"ArchDiagram","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arch-review-v1","name":"Architectural Code Review v1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arcs-text-to-sql","name":"ARCS","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","23 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-6 Sol (#13)."],["scores.headroom","pass","The best is 57.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on megagonlabs.github.io: 28 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arctic-extract-business-documents","name":"Arctic-Extract Business Document Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"are-bench","name":"ARE-bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 66.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arena-expert","name":"Arena Expert","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (90 entries)."],["scores.models","pass","64 models we track, plus 26 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for GLM-4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on arena.ai: 72 rows for models we track and 26 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arena-search-factuality","name":"Arena Search Factuality","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","7 models we track, plus 25 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on arena.ai: 7 rows for models we track and 25 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arena-hard-v2","name":"Arena-Hard v2.0","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries), launch posts (20 scores) and system cards (14 scores)."],["scores.models","pass","47 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Granite 4.2 30B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","info","Saturated. The best is 90.2%, past 85% since April 2025."],["import.importer","pass","We read the board on github.com: 26 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arena-hard-auto","name":"Arena-Hard-Auto","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (70 entries), launch posts (26 scores) and system cards (25 scores)."],["scores.models","pass","79 models we track, plus 28 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Granite 4.1 30B. That’s expected for a test replaced by Arena-Hard v2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-R1 (#130)."],["scores.headroom","info","Saturated. The best is 97.1%, past 85% since September 2024."],["import.importer","pass","We read the board on github.com: 42 rows for models we track and 28 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arena-t2i-hard","name":"Arena-T2I Hard","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","11 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on banyuanhao.github.io: 17 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arena-webdev-correctness","name":"ArenaWebdevCorrectness","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 81.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arena-webdev-preferences","name":"ArenaWebdevPreferences","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Luna (#36)."],["scores.headroom","pass","The best is 69.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"argo-bench","name":"Argo-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","24 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on argo-bench.com: 24 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arguana-counterargs","name":"ArguAna Counterargs","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"argus-inspection","name":"Argus Inspection","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (26 entries)."],["scores.models","pass","24 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Claude Sonnet 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 24 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"argus-bench-v2","name":"Argus-Bench v2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Saturated. The best is 85.7%, past 85% since September 2025."],["import.importer","pass","We read the board on kaggle.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aria-digital-twin","name":"Aria Digital Twin","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"proof-news-arizona-bilingual-voter-questions","name":"Arizona voter questions in English and Spanish","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Gemini 1.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on datawrapper.dwcdn.net: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arkitscenes","name":"ARKitScenes","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Qwen3-VL-235B-A22B. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 53.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"armnetbench","name":"ArmnetBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for MolmoAct 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 47.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on armnet-dev.github.io: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arpa-armenian-paraphrase-corpus","name":"ARPA Armenian Paraphrase Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"artcap","name":"ArtCap","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"artedit-bench-eval","name":"ArtEdit-Bench-Eval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#112)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"artedit-bench-lr","name":"ArtEdit-Bench-Lr","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","5 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arti4d","name":"Arti4D","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"articulate3d","name":"Articulate3D","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on insait-institute.github.io: 12 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"artifactarena","name":"ArtifactArena","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","21 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on artifactarena.ai: 59 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"artifactsbench","name":"ArtifactsBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries), launch posts (5 scores) and system cards (4 scores)."],["scores.models","pass","24 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Solar Open 2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 72.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on artifactsbenchmark.github.io: 21 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"artifactsbench-gemini-0506","name":"ArtifactsBench (Gemini 2.5 Pro 0506 judge)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","26 models we track, plus 5 other entries on its leaderboard."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Qwen3-30B-A3B-Instruct. That’s expected for a test replaced by ArtifactsBench."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 57%, under 85%, so it still separates models."],["import.importer","pass","We read the board on artifactsbenchmark.github.io: 28 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-wer","name":"Artificial Analysis average word error rate","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.5 Transcribe."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-coding-agent-index-v1-0","name":"Artificial Analysis Coding Agent Index v1.0","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-coding-agent-index-v1-1","name":"Artificial Analysis Coding Agent Index v1.1","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-coding-agent-index-v1-4","name":"Artificial Analysis Coding Agent Index v1.4","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-coding-agent-index-v1-5","name":"Artificial Analysis Coding Agent Index v1.5","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"artificial-analysis-coding-agent-index","name":"Artificial Analysis Coding Agent Index v2","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-cyber-index","name":"Artificial Analysis Cyber Index","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Mistral Large 4 (Le Chonk, open weights)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 51%, under 85%, so it still separates models."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-decision-index","name":"Artificial Analysis Decision Index","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-image-arena","name":"Artificial Analysis Image Arena","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-intelligence-index-v1","name":"Artificial Analysis Intelligence Index v1","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-intelligence-index-v2","name":"Artificial Analysis Intelligence Index v2","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GLM-4.5-Air. That’s expected for a test replaced by Artificial Analysis Intelligence Index v2.2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-intelligence-index-v2-2","name":"Artificial Analysis Intelligence Index v2.2","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-intelligence-index-v3","name":"Artificial Analysis Intelligence Index v3","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 13 months ago, in October 2025, for MiniMax-M2. That’s expected for a test replaced by Artificial Analysis Intelligence Index v4.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-intelligence-index-v4-0","name":"Artificial Analysis Intelligence Index v4.0","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for MiniMax-M2.5. That’s expected for a test replaced by Artificial Analysis Intelligence Index v4.1."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-intelligence-index-v4-1","name":"Artificial Analysis Intelligence Index v4.1","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (7 scores) and system cards (3 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3-Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-intelligence-index-v4-2","name":"Artificial Analysis Intelligence Index v4.2","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-intelligence-index-v4-3","name":"Artificial Analysis Intelligence Index v4.3","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (4 scores) and system cards (2 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Muse Wonder (#16)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"artificial-analysis-intelligence-index","name":"Artificial Analysis Intelligence Index v5","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (84 entries), launch posts (7 scores) and system cards (3 scores)."],["scores.models","pass","70 models we track, plus 15 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on Artificial Analysis: 141 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 6 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-progress-index","name":"Artificial Analysis Progress Index","summary":{"pass":1,"warn":3,"fail":0,"info":6,"text":"1 of 4 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-search-index","name":"Artificial Analysis Search Index","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-speech-arena","name":"Artificial Analysis Speech Arena","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemini 3.1 Flash TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aa-video-arena","name":"Artificial Analysis Video Arena","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"artificial-turf-war","name":"Artificial Turf War","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from August 2026, for DeepSeek V4 Pro 0813."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on artificialturfwar.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arxivcs-matharena","name":"ArXivCS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arxivlean","name":"ArXivLean","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries) and system cards (1 score)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 79.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on matharena.ai: 10 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arxivmath","name":"ArXivMath","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries), launch posts (3 scores) and system cards (6 scores)."],["scores.models","pass","15 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","6 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Saturated. The best is 94.3%, past 85% since July 2026."],["import.importer","pass","We read the board on matharena.ai: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arxivphys-matharena","name":"ArXivPhys","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arxivrollbench","name":"ArxivRollBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","31 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 70.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxivroll.moreoverai.com: 31 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asag2024","name":"ASAG2024","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ascend","name":"ASCEND","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ascendgym","name":"AscendGYM","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on slai-aitp.github.io: 21 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ascii-bench-botterill","name":"ASCII Bench (Jason Botterill)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on asciibench.com: 13 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asdiv","name":"ASDiv","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asfx-sed","name":"ASFX-SED","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asgardbench","name":"AsgardBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","7 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Kimi K2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 76.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asi-bench","name":"ASI-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 18 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asian-language-treebank","name":"Asian Language Treebank","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asimov","name":"ASIMOV","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 97.9%, past 85% since April 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asimov-1","name":"ASIMOV v1","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 1 entry at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asimov-agentic","name":"ASIMOV-Agentic","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from July 2026, for Gemini Robotics ER 2."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asimov-jailbreak","name":"ASIMOV-Jailbreak","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ask-norwegian-grammatical-error-correction","name":"ASK Norwegian Grammatical Error Correction","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ask-ubuntu-duplicate-question-retrieval","name":"Ask Ubuntu duplicate question retrieval","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (256 entries)."],["scores.models","pass","5 models we track, plus 251 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MTEB: 5 rows for models we track and 251 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asl-4-virology-uplift-trial-anthropic","name":"ASL-4 virology uplift trial (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Mythos Preview. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 66.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asper-bench","name":"Asper-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aspi-scale","name":"ASPI","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aspire","name":"Aspire","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5-9B (#128)."],["scores.headroom","pass","The best is 25.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"assembly101-o","name":"Assembly101-O","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"assemblyai-speech-benchmarks","name":"AssemblyAI Speech Benchmarks","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","3 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.5 Transcribe."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on assemblyai.com: 3 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"assertionforge-designs","name":"AssertionForge design evaluation","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"assertllm2","name":"AssertLLM2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 11 months ago, in December 2025, for GPT-5.2. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Saturated. The best is 95%, past 85% since April 2025."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"assetopsbench","name":"AssetOpsBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Granite 3.3 8B Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4.1 (#138)."],["scores.headroom","pass","The best is 65%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"assin","name":"ASSIN","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"assin-2","name":"ASSIN 2","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on docs.google.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"assistantbench","name":"AssistantBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on assistantbench.github.io: 1 entry at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"assurai","name":"AssurAI","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"astabench","name":"AstaBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","14 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 58%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 20 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"astra-bench","name":"ASTRA-bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"astro-qa","name":"Astro-QA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"astroclimb","name":"AstroCLIMB","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 19 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"astrollava-image-and-text-retrieval","name":"AstroLLaVA image and text retrieval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"astromembench","name":"AstroMemBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"astromlab-1","name":"AstroMLab 1","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (59 entries)."],["scores.models","pass","24 models we track, plus 35 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Qwen2.5-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 82.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 24 rows for models we track and 35 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"astrovisbench","name":"AstroVisBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Claude Opus 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4 (#109)."],["scores.headroom","pass","The best is 49.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on astrovisbench.github.io: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"astrovlbench","name":"AstroVLBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Intern-S1-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 68.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asynctool","name":"AsyncTool","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries)."],["scores.models","pass","9 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for DeepSeek-V3.1-Terminus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 38.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 25 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"atepp","name":"ATEPP","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on tangjjbetsy.github.io: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"athenabench","name":"AthenaBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","6 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3-Coder-Next."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 66.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 6 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"exa-atlas","name":"ATLAS (Exa)","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on exa.ai: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"atlas-opencompass","name":"ATLAS (OpenCompass)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 42.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"atlas-finance","name":"ATLAS-Finance","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on joinhandshake.com: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"atlas-finance-1-1","name":"ATLAS-Finance v1.1.0","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 10.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on joinhandshake.com: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"atm-bench","name":"ATM-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","13 models we track, plus 16 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 86%, past 85% since August 2025."],["import.importer","pass","We read the board on atmbench.github.io: 27 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"atom-bench","name":"ATOM-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","2 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GR00T N1.6."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 48.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on flageval-baai.github.io: 2 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"atomic-challenges-irregular","name":"Atomic Challenges","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry), launch posts (3 scores) and system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6 Cyber."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2026."],["import.importer","pass","We read the board on irregular.com: 1 row for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"atrex-bench","name":"Atrex-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Qwen3.7-Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"attaq","name":"AttaQ","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Granite 4.1 30B. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 88.5%, past 85% since February 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audio-arena-arcada","name":"Audio Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","8 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Grok Voice 3.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on audioarena.ai: 16 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audio-arithmetic-24-music-workflow-checks","name":"Audio Arithmetic: 24 Music Workflow Checks","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 13 months ago, in October 2025, for Claude Haiku 4.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Haiku 4.5 (#110)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since August 2025."],["import.importer","pass","We read the board on kaggle.com: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audio-mc","name":"Audio MC","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (25 entries), launch posts (5 scores) and system cards (4 scores)."],["scores.models","pass","23 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","pass","The best is 71.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 30 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 4 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audio-pairs","name":"Audio PAIRS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audio-reasoning-challenge-2026","name":"Audio Reasoning Challenge 2026","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","27 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on audio-reasoning-challenge.github.io: 27 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audio-understanding-benchmark-mistral","name":"Audio Understanding Benchmark (Mistral)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Voxtral Small. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 86.6%, past 85% since July 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audio-ntrex-4l","name":"Audio-NTREX-4L","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audio-visual-event-localization","name":"Audio-Visual Event Localization","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","15 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 15 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audiobench","name":"AudioBench","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audiocaps","name":"AudioCaps","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries)."],["scores.models","pass","17 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on audiocaps.github.io: 17 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audiojevbench","name":"AudioJevBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","4 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Qwen3-Omni-30B-A3B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on benchmarkheaven.com: 4 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audiomnist","name":"AudioMNIST","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audiomos-challenge-2025","name":"AudioMOS Challenge 2025","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on docs.google.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audiosep-eval-benchmark","name":"AudioSep Eval Benchmark","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on audio-agi.github.io: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audioset-strong","name":"AudioSet Strong","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audiotrust","name":"AudioTrust","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","4 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#112)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audio-video-bench-tencent","name":"AudioVideoBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prism-multi-turn-coding-augment","name":"Augment Prism multi-turn coding benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on augmentcode.com: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"augmented-reality-patent-search","name":"Augmented Reality Patent Search","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"au-bas-reconciliation-v1","name":"Australian BAS Reconciliation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autoadvexbench","name":"AutoAdvExBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 22%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autobaxbench","name":"AutoBaxBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","18 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","5 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 88.6%, past 85% since November 2026."],["import.importer","pass","We read the board on baxbench.com: 18 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autobench-testbench","name":"AutoBench (HDL testbench generation)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","pass","The best is 71.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autobench-agronomy","name":"AutoBench Agronomy LLM Benchmark","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","34 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Claude Opus 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 35 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autobencher","name":"AutoBencher","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries)."],["scores.models","pass","9 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for GPT-4 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#172)."],["scores.headroom","pass","The best is 55%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autocad-bench","name":"AutoCAD Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 46%, under 85%, so it still separates models."],["import.importer","pass","We read the board on markovstudios.com: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autocast","name":"Autocast","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autocfd-incompressible-flow","name":"AutoCFD Incompressible Flow Benchmark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","2 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for Llama 3.3 70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 31.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autocodebench-tencent","name":"AutoCodeBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (38 entries)."],["scores.models","pass","34 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GLM-4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 52.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on autocodebench.github.io: 41 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autods-scientific-cases","name":"AutoDS scientific machine learning cases","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autoelicit","name":"AutoElicit","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries) and system cards (1 score)."],["scores.models","pass","6 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on osu-nlp-group.github.io: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autoeval","name":"AutoEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for OpenVLA."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 40%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autofactory-bench","name":"AutoFactory-Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 72.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on autofactorybench.com: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autoform-hard","name":"Autoform-Hard","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 87.5%, past 85% since November 2026."],["import.importer","pass","We read the board on vals.ai: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autoformbench","name":"AutoformBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 82.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autointent-bio-xai","name":"Autointent-Bio","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autointent-chem-xai","name":"Autointent-Chem","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autolab","name":"AutoLab","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on autolab.moe: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autologi","name":"AutoLogi","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries), launch posts (2 scores) and system cards (8 scores)."],["scores.models","pass","10 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Kimi K2. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K2 (#117)."],["scores.headroom","info","Saturated. The best is 89.5%, past 85% since April 2025."],["import.importer","pass","We read the board on raw.githubusercontent.com: 2 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"automated-llm-speedrunning-benchmark-meta","name":"Automated LLM Speedrunning Benchmark","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 46%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 50 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asap-essay-scoring-2","name":"Automated Student Assessment Prize 2.0","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2,708 entries)."],["scores.models","pass","2 models we track, plus 2,706 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in October 2024, for π0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 2 rows for models we track and 2,706 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asap-essay-scoring","name":"Automated Student Assessment Prize: Automated Essay Scoring","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (154 entries)."],["scores.models","pass","1 model we track, plus 153 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in November 2023, for Monkey. That’s expected for a test replaced by Automated Student Assessment Prize 2.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 1 row for models we track and 153 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"automationbench","name":"AutomationBench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (46 entries), launch posts (54 scores) and system cards (19 scores)."],["scores.models","pass","76 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 73.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on zapier.com: 150 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 21 pairs, each explained in our notes."],["consistency.issues","warn","1 finding disputed or fixed only in a copy: Unclear tasks and broken grader."]]},{"id":"automedbench","name":"AutoMedBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"automotive-env","name":"Automotive-ENV","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","9 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-122B-A10B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 49.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on automotive-env.github.io: 9 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ara-evaluations-anthropic","name":"Autonomous Replication and Adaption (ARA) evaluations (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in March 2024, for Claude 3 Opus. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autonomy-10-hud","name":"Autonomy-10","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autopatchbench","name":"AutoPatchBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Llama 4 Maverick."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 11.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on engineering.fb.com: 3 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autopenbench","name":"AutoPenBench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autoplc","name":"AutoPLC","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Qwen2.5-Coder (7B)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-8B (#203)."],["scores.headroom","pass","The best is 9.88%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autoresearch","name":"AutoResearch","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autoresearchbench","name":"AutoResearchBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","13 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 9.39%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cheryou.github.io: 13 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autoresearchexam","name":"AutoResearchExam","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on benchmarks.bespokelabs.ai: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"autovend-stepfun","name":"AutoVend","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"av-safetybench","name":"AV-SafetyBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for LTX-2.3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on miko-mitama.github.io: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"av-speakerbench","name":"AV-SpeakerBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","8 models we track, plus 17 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 77.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on plnguyen2908.github.io: 5 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"avalanchebench","name":"AvalancheBench (May study)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"avalonbench","name":"AvalonBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on avalonbench.github.io: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"avcc-next-speaker-prediction","name":"AVCC Next-Speaker Prediction","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"average-in-eval-hack-rate-anthropic","name":"Average in-eval reward hacking rate (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 13 months ago, in October 2025, for Claude Haiku 4.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Sonnet 4.5 (#81)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"avgen-bench","name":"AVGen-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","5 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for LTX-2.3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on microsoft.github.io: 5 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aviation-blind-spot-benchmark","name":"Aviation Blind Spot Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 0%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"avmeme-exam","name":"AVMeme Exam","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries) and launch posts (1 score)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Seed2.0 Lite (0428)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 81.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on avmemeexam.github.io: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"avps-360","name":"AVPS-360","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"avut","name":"AVUT","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries), launch posts (3 scores) and system cards (2 scores)."],["scores.models","pass","11 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","info","Saturated. The best is 86.6%, past 85% since March 2026."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aws-bench","name":"aws-bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aya-redteaming","name":"Aya Red-teaming","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ayavisionbench","name":"AyaVisionBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (1 score)."],["scores.models","pass","2 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Aya Vision 32B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 64%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cohere.com: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"b-moca","name":"B-MoCA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for Gemini 1.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 47.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ba-calendar","name":"BA-Calendar","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Phi-4-Reasoning-plus."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 67.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"baanchmark","name":"BAAnchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (62 entries)."],["scores.models","pass","57 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on dubovskylabs.com: 63 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"baba-is-ai","name":"Baba Is AI","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","pass","The best is 43.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on BALROG: 13 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"babaisbench","name":"BabaIsBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","32 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since February 2026."],["import.importer","pass","We read the board on quesma.com: 32 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"babe","name":"BABE","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","10 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Seed2.0 Lite (0428)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 57.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"babel-imagenet","name":"Babel-ImageNet","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 12 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"babelarena","name":"BabelArena","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"babi-qa-tasks","name":"bAbI QA Tasks","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"babilong","name":"BABILong","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (30 entries)."],["scores.models","pass","6 models we track, plus 24 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for Phi-3-medium-128k-instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 72.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track and 24 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"babyai","name":"BabyAI","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","5 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in December 2023, for Mistral-7B-Instruct-v0.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 2 (#192)."],["scores.headroom","pass","The best is 37.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on AgentBoard: 5 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"babyvision","name":"BabyVision","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries), launch posts (25 scores) and system cards (9 scores)."],["scores.models","pass","45 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 95.4%, past 85% since June 2026."],["import.importer","pass","We read the board on unipat.ai: 28 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 9 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"backendbench-meta","name":"BackendBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"backrooms-3d-clue-trail-navigation","name":"Backrooms: 3D Clue-Trail Navigation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"backsearch","name":"BackSearch","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"backtracing","name":"Backtracing","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"balatro-bench-andrzejewski","name":"Balatro Bench (Michael Andrzejewski)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 21 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"balatrobench","name":"BalatroBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","16 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Kimi K2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on balatrobench.com: 16 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ballotbench","name":"BallotBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ballotbench.com: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"balrog","name":"BALROG","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries)."],["scores.models","pass","34 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 68.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on balrogai.com: 36 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bamboogle","name":"Bamboogle","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Cohere Command A."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 76%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"banglaser","name":"BanglaSER","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bank-question-corpus","name":"Bank Question Corpus","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (101 entries)."],["scores.models","pass","4 models we track, plus 97 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Qwen3-Embedding-0.6B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mteb-leaderboard.hf.space: 4 rows for models we track and 97 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bankertoolbench","name":"BankerToolBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries) and launch posts (3 scores)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 78.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bankertoolbench-1-1","name":"BankerToolBench v1.1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 32.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on joinhandshake.com: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"banking77","name":"BANKING77","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (272 entries)."],["scores.models","pass","5 models we track, plus 267 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 92.3%, past 85% since June 2025."],["import.importer","pass","We read the board on MTEB: 5 rows for models we track and 267 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"banspeech","name":"BanSpeech","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on Bengali ASR Benchmark: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"barc-bench","name":"BARC-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 72.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"barec-shared-task-2025","name":"BAREC Shared Task 2025","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","16 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 16 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"barec-shared-task-2026","name":"BAREC-ST-2026","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"barterbench","name":"BarterBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"base64bench","name":"Base64Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 75.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on lesswrong.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bash-agent-evals-braintrust","name":"Bash Agent Evals","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on braintrust.dev: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"basharena","name":"BashArena","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (1 entry) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bashbench","name":"BashBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on bashcontrol.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"basic-reasoning-with-letters","name":"Basic reasoning with letters","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","11 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is Claude Sonnet 5.5 (#9)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since February 2026."],["import.importer","pass","We read the board on kaggle.com: 11 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"basqueglue","name":"BasqueGLUE","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"baxbench","name":"BaxBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries)."],["scores.models","pass","34 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Claude Opus 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","pass","The best is 68.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on baxbench.com: 105 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bazaar-lech-mazur","name":"BAZAAR","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (53 entries)."],["scores.models","pass","24 models we track, plus 29 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 26 rows for models we track and 29 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"big-bench-extra-hard-micro","name":"BBEH micro average","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries), launch posts (9 scores) and system cards (13 scores)."],["scores.models","pass","23 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","pass","The best is 74.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 11 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bbh-ja","name":"BBH-ja","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bbq-bias-benchmark-for-qa","name":"BBQ (Bias Benchmark for QA)","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (25 scores)."],["scores.models","pass","25 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 96.6%, past 85% since May 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bcsc-self-assessment-program","name":"BCSC Self-Assessment Program ophthalmology questions","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bdslw401","name":"BdSLW401","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bdslw60","name":"BdSLW60","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bea-2023-generating-ai-teacher-responses","name":"BEA 2023 Shared Task on Generating AI Teacher Responses in Educational Dialogues","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","1 model we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in June 2023, for GPT-4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on sig-edu.org: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bea-2025-pedagogical-ability-assessment","name":"BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutors","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on sig-edu.org: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"beam-long-term-memory","name":"BEAM","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Muse Glimmer."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"beancount-ledger","name":"Beancount Ledger","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"beans-zero","name":"BEANS-Zero","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bear","name":"BEAR","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (21 entries)."],["scores.models","pass","4 models we track, plus 17 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Gemma 2B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 7B (#211)."],["scores.headroom","pass","The best is 65.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 5 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bedside-bench","name":"Bedside Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","18 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 94.6%, past 85% since July 2026."],["import.importer","pass","We read the board on fireworks.ai: 18 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"beemo","name":"Beemo","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","4 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for Qwen2-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Falcon-7B (#246)."],["scores.headroom","pass","The best is 74.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 28 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"behavior-challenge-2025","name":"BEHAVIOR Challenge 2025","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","The newest came 11 months ago, in December 2025, for GR00T N1.6. That’s expected for a test replaced by BEHAVIOR Challenge 2026."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on behavior.stanford.edu: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"behavior-challenge-2026","name":"BEHAVIOR Challenge 2026","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 6 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"behavior-1k","name":"BEHAVIOR-1K","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 6 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"behavior2code-turing","name":"Behavior2Code","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 31.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.turing.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"behnke-geiger-work-center-scheduling","name":"Behnke and Geiger Flexible Job Shop Instances","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"beir-fiqa-2018","name":"BEIR FiQA-2018 Retrieval","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (295 entries)."],["scores.models","pass","5 models we track, plus 290 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on leaderboard.mteb.org: 5 rows for models we track and 290 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"belarusianglue","name":"BelarusianGLUE","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"belebele","name":"Belebele","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries) and system cards (1 score)."],["scores.models","pass","6 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Glimmer."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-3.5 Turbo (#200)."],["scores.headroom","info","Saturated. The best is 87.8%, past 85% since October 2026."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ben-10-regional-asr","name":"Ben-10 Regional ASR","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on huggingface.co: 1 entry at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bench-on-the-clocktower","name":"Bench on the Clocktower","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 58%, under 85%, so it still separates models."],["import.importer","pass","We read the board on lesswrong.com: 10 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"btf-1","name":"Bench to the Future","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a Brier score, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bench4hls","name":"Bench4HLS","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 50%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"benchcad","name":"BenchCAD","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (5 scores) and system cards (6 scores)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 95.9%, past 85% since September 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"benchcad-2-0","name":"BenchCAD 2.0","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 59%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchcad.com: 25 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"benchcad-2-1","name":"BenchCAD 2.1","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 83.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchcad.com: 70 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"benchcad-2-1-beta","name":"BenchCAD 2.1-beta","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"benchmax","name":"BenchMAX","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","22 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Gemma 3 1B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 22 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"benczechmark","name":"BenCzechMark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (78 entries)."],["scores.models","pass","40 models we track, plus 38 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Muse Glimmer."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek V4 Flash 0731 (#52)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 41 rows for models we track and 38 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bengali-classical-music-generation-kader","name":"Bengali Classical Music Generation","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","4 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 40%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"benger","name":"BenGER","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"berkeley-function-calling-leaderboard-bfcl","name":"Berkeley Function Calling Leaderboard (BFCL)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (89 entries), launch posts (28 scores) and system cards (28 scores)."],["scores.models","pass","97 models we track, plus 22 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3-235B-A22B-Thinking (#108)."],["scores.headroom","info","Saturated. The best is 88.5%, past 85% since July 2024."],["import.importer","pass","We read the board on github.com: 87 rows for models we track and 22 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 11 pairs differs by more than 10 points, the largest for Phi-4 Mini (+39 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bertaqa","name":"BertaQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (39 entries)."],["scores.models","pass","26 models we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for Gemma 2 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 81.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 27 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"betterchartqa","name":"BetterChartQA","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 72.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"beyond-aime","name":"Beyond AIME","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries), launch posts (13 scores) and system cards (8 scores)."],["scores.models","pass","15 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Mistral Medium 3.5 (#116)."],["scores.headroom","info","Saturated. The best is 87%, past 85% since February 2026."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"beyond-survival","name":"Beyond Survival","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","13 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for DeepSeek-V3.2-Exp."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 58.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 13 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bfcl-v4","name":"BFCL V4","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (80 entries), launch posts (31 scores) and system cards (11 scores)."],["scores.models","pass","95 models we track, plus 15 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.7-Max (#49)."],["scores.headroom","pass","The best is 77.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on gorilla.cs.berkeley.edu: 94 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 1 pair differs by more than 10 points, the largest for Nova 2 Lite (+33 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bgglue","name":"bgGLUE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on bgglue.github.io: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bhashabench-krishi","name":"BhashaBench-Krishi","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (31 entries)."],["scores.models","pass","14 models we track, plus 17 other entries on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for Gemma 3 270M. That’s expected for a test replaced by BhashaBench-Multi (Krishi)."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is gpt-oss-120b (#126)."],["scores.headroom","pass","The best is 72.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 14 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bhashabench-multi-krishi","name":"BhashaBench-Multi (Krishi)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bhl-ocr-eval","name":"BHL OCR Eval","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","8 models we track, plus 18 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-OCR-1.0."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5-9B (#128)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on huggingface.co: 8 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bhs-syntactic-evaluation","name":"BHS Syntactic Evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bibimbap","name":"BIBIMBAP","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 50.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on nbharathik.github.io: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bible-accuracy-benchmark-youversion","name":"Bible Accuracy Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bible-translation-meta","name":"BIBLE Translation Benchmark (Meta)","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biblically-informed-bot-learning-evaluation","name":"Biblically Informed Bot Learning Evaluation","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","18 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Gemma 4 E2B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 52.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 18 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"big-bench-audio","name":"Big Bench Audio","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (7 scores) and system cards (1 score)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from October 2026, for Grok Voice 3.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 98.5%, past 85% since March 2026."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bigcode-models-leaderboard","name":"Big Code Models Leaderboard","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (60 entries)."],["scores.models","pass","9 models we track, plus 51 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Qwen2.5-Coder-32B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-Coder-32B (#188)."],["scores.headroom","pass","The best is 59.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 9 rows for models we track and 51 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"big-sleep-evaluation-google","name":"Big Sleep Evaluation (Google)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries) and launch posts (1 score)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.5 Flash Cyber."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.5 Flash (#44)."],["scores.headroom","pass","The best is 72%, under 85%, so it still separates models."],["import.importer","pass","We read the board on deepmind.google: 3 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"big-bench","name":"BIG-bench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"big-bench-abstract-narrative-understanding","name":"BIG-bench Abstract Narrative Understanding","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for Gemma 3 270M."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 82.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ADAGE: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"big-bench-english-proverbs","name":"BIG-bench English Proverbs","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (51 entries)."],["scores.models","pass","2 models we track, plus 49 other entries on its leaderboard."],["scores.recent","warn","The newest came 4 years ago, in December 2021, for Gopher (280B)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 57.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 49 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"big-bench-extra-hard","name":"BIG-Bench Extra Hard","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Gemma 3 12B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3-mini (#124)."],["scores.headroom","pass","The best is 44.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 11 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"big-bench-hard","name":"BIG-Bench Hard","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries), launch posts (47 scores), system cards (57 scores) and claims Epoch AI collected (45 models)."],["scores.models","pass","100 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Beam 1.0 Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Saturated. The best is 94.3%, past 85% since March 2024."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"big-bench-understanding-fables","name":"BIG-bench Understanding Fables","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (51 entries)."],["scores.models","pass","2 models we track, plus 49 other entries on its leaderboard."],["scores.recent","warn","The newest came 4 years ago, in December 2021, for Gopher (280B)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 39.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 49 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bigcodearena","name":"BigCodeArena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for GLM-4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3.2-Exp (#99)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 14 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bigcodebench","name":"BigCodeBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (120 entries), launch posts (8 scores) and system cards (9 scores)."],["scores.models","pass","83 models we track, plus 51 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","pass","The best is 60.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on bigcode-bench.github.io: 70 rows for models we track and 51 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bigcodebench-hard","name":"BigCodeBench-Hard","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (155 entries), launch posts (3 scores) and system cards (2 scores)."],["scores.models","pass","99 models we track, plus 59 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Seed-Coder-8B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#104)."],["scores.headroom","pass","The best is 35.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on bigcode-bench.github.io: 104 rows for models we track and 59 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bigearthnet-txt","name":"BigEarthNet.txt","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","3 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 0.79%, under 85%, so it still separates models."],["import.importer","pass","We read the board on txt.bigearth.net: 3 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bigfinancebench","name":"BigFinanceBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries) and launch posts (2 scores)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Terra."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Terra (#18)."],["scores.headroom","pass","The best is 58.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biglaw-bench","name":"BigLaw Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries) and launch posts (3 scores)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Opus 4.7. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 91%, past 85% since February 2026."],["import.importer","pass","We read the board on harvey.ai: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biglaw-bench-retrieval","name":"BigLaw Bench Retrieval","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biglaw-bench-spa-deal-points","name":"BigLaw Bench Workflows: SPA Deal Points","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on harvey.ai: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biglaw-bench-arena","name":"BigLaw Bench: Arena","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on harvey.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biglaw-bench-global","name":"BigLaw Bench: Global","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biglaw-bench-research","name":"BigLaw Bench: Research","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bigobench","name":"BigO(Bench)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","18 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Code World Model (CWM)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is QwQ-32B (#137)."],["scores.headroom","pass","The best is 41.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on facebookresearch.github.io: 18 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bol-bench-okrapdf","name":"Bill of Lading OCR Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","13 models we track, plus 11 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.7 Flash (#26)."],["scores.headroom","info","Saturated. The best is 94.9%, past 85% since January 2025."],["import.importer","pass","We read the board on okrapdf.com: 13 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"billsum","name":"BillSum","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bim-bench","name":"BIM Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on bimbench.com: 36 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bim-edit","name":"BIM-Edit","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 87.7%, past 85% since September 2026."],["import.importer","pass","We read the board on ONESTRUCTION: 20 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bimanual-chore-scheduling","name":"Bimanual Chore Scheduling","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"binary-exploitation-benchmark-anthropic-oss-fuzz-based","name":"Binary Exploitation Benchmark (Anthropic, OSS-Fuzz based)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 91.1%, past 85% since September 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"binary-reasoning-v1","name":"Binary Reasoning v1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"binaryaudit","name":"BinaryAudit","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 49.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on quesma.com: 26 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"algorithmwatch-bing-chat-elections-2023","name":"Bing Chat election answers in Switzerland, Bavaria, and Hesse","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bingo-visual-hallucination","name":"Bingo (Visual Hallucination)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bio-static-jailbreak-eval-google","name":"Bio Static Jailbreak Eval","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bio-search-l2","name":"Bio-Literature Search Level 2","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bio-ppi-risk-serous-vs-endometrioid","name":"Bio-PPI Risk: Serous vs Endometrioid","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bio-probe-openai","name":"Bio-Probe","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a Brier score, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bio-vqa-imageomics","name":"Bio-VQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Gemma 4 12B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on smrayeed.github.io: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bioasq","name":"BioASQ","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biobasicsbench","name":"BioBasicsBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 66.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openai.com: 7 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biobench-bytedance","name":"BIOBench (ByteDance Seed)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Seed1.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 53.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biochemistry-knowledge-improvement-openai","name":"Biochemistry knowledge improvement","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for GPT-5.5 Pro. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.5 Pro (#15)."],["scores.headroom","pass","The best is 39.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biocircuit","name":"BioCircuit","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bc5cdr","name":"BioCreative V Chemical-Disease Relation Task","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biocreative-vii-litcovid","name":"BioCreative VII LitCovid","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biocreative-viii-dysmorphology","name":"BioCreative VIII Task 3 Genetic Phenotype Normalization from Dysmorphology Physical Examinations","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codalab.lisn.upsaclay.fr: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biodivner","name":"BiodivNER","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bioinformatics-evaluations-anthropic","name":"Bioinformatics evaluations (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Claude Opus 4.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","pass","The best is 73.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biological-safety-evals-openai","name":"Biological safety evaluations (OpenAI)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bio-static-hazard-eval-google","name":"Biological static hazard evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bio-threat-creation-uplift-study-openai","name":"Biological threat creation uplift study","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bwt-lfqa","name":"Biological Wastewater Treatment Long-Form QA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biology-refusal-evals-openai","name":"Biology Model Refusal Evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 99.9%, past 85% since November 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biology-instructions","name":"Biology-Instructions","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Intern-S2-397B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 55.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biolp-bench","name":"BioLP-bench","summary":{"pass":5,"warn":1,"fail":0,"info":4,"text":"5 of 6 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (9 scores)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Grok 4.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4 (#86)."],["scores.headroom","pass","The best is 49%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biomaze","name":"BioMaze","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-8B (#203)."],["scores.headroom","pass","The best is 61.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biomedarena-datatecnica","name":"BiomedArena","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biomedical-image-analysis-anthropic","name":"Biomedical image analysis (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 72.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bioml-bench","name":"BioML-bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biomni-eval1","name":"Biomni-Eval1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biomnibench","name":"BiomniBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on xbench.org: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biomysterybench","name":"BioMysteryBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries), launch posts (8 scores) and system cards (5 scores)."],["scores.models","pass","39 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","8 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 90.3%, past 85% since August 2026."],["import.importer","pass","We read the board on Vals AI: 33 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bionne-2024","name":"BioNNE 2024","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codalab.lisn.upsaclay.fr: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bionne-l","name":"BioNNE-L","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codalab.lisn.upsaclay.fr: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bionne-r","name":"BioNNE-R","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codabench.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biopharma-bench","name":"Biopharma Bench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","8 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 65.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on eval.raycaster.ai: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biophys-bridge","name":"BioPhys-Bridge","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biopipelinebench-anthropic","name":"BioPipelineBench (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Saturated. The best is 87.7%, past 85% since May 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biopipelinebench-verified-anthropic","name":"BioPipelineBench Verified (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Opus 4.7. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.7 (#31)."],["scores.headroom","pass","The best is 83.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biored","name":"BioRED","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bioriskeval","name":"BioRiskEval","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biosecbench-function","name":"BioSecBench-Function","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries) and system cards (1 score)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 73.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 42 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biosecbench-refusal","name":"BioSecBench-Refusal","summary":{"pass":11,"warn":1,"fail":0,"info":0,"text":"11 of 12 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries) and system cards (1 score)."],["scores.models","pass","22 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 62.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 31 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biosecbench-surveillance","name":"BioSecBench-Surveillance","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries) and system cards (1 score)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 60.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 42 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biosecurity-vct-xai","name":"Biosecurity VCT","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","pass","The best is 47.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biosimbench","name":"BioSimBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","12 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on celltype.com: 12 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biotier-securebio","name":"BioTIER","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biotier-securebio-permit","name":"BioTIER Permit","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (77 entries) and system cards (1 score)."],["scores.models","pass","73 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since August 2024."],["import.importer","pass","We read the board on securebio.org: 73 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biotier-securebio-refuse","name":"BioTIER Refuse","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (77 entries) and system cards (4 scores)."],["scores.models","pass","75 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 99.1%, past 85% since September 2025."],["import.importer","pass","We read the board on securebio.org: 73 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biotier-v1-refuse-securebio","name":"BioTIER v1 Refuse","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Saturated. The best is 98%, past 85% since June 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biotool-wet-lab-actions-openai","name":"Biotool and Wet Lab Actions","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 22 months ago, in January 2025, for o3-mini. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biousebench-xai","name":"BioUseBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","info","Saturated. The best is 91.4%, past 85% since August 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"biovita-retrieval-benchmark","name":"BioVITA retrieval benchmark","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bioweapons-acquisition-uplift-trial-anthropic","name":"Bioweapons acquisition uplift trial (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Claude Opus 4. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4 (#109)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bioweapons-knowledge-questions-anthropic","name":"Bioweapons knowledge questions","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Claude Opus 4. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4 (#109)."],["scores.headroom","pass","The best is 51.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bird-critic","name":"BIRD-CRITIC","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for DeepSeek-V3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-R1 (#130)."],["scores.headroom","pass","The best is 35.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on bird-critic.github.io: 10 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bird-interact","name":"BIRD-Interact","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 25.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 10 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bird-python","name":"BIRD-Python","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","8 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Qwen3-Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-Max (#111)."],["scores.headroom","pass","The best is 73.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bird-sql","name":"Bird-SQL","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (120 entries), launch posts (7 scores) and system cards (10 scores)."],["scores.models","pass","25 models we track, plus 107 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 72.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on bird-bench.github.io: 41 rows for models we track and 82 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"birdset","name":"BirdSet","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 16 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"birdsnap","name":"Birdsnap","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"birthdayfacts-openai","name":"BirthdayFacts","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 2 years ago, in September 2024, for o1-mini. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"anes-feeling-thermometers-bisbee","name":"Bisbee ANES feeling thermometers","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bixbench","name":"BixBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries), launch posts (1 score) and system cards (3 scores)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 93.8%, past 85% since July 2026."],["import.importer","pass","We read the board on raw.githubusercontent.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bixbench-hypothesis","name":"BixBench-Hypothesis","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bixbench-3","name":"BixBench3","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on advances.edisonscientific.com: 20 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bizbench-ant","name":"BizBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Sol."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 70%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bizfinbench","name":"BizFinBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","15 models we track, plus 10 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Qwen3-1.7B. That’s expected for a test replaced by BizFinBench v2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 73.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 15 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bizfinbench-2","name":"BizFinBench v2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","9 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 61.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on hithink-research.github.io: 9 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"black-box-rna-sequence-design-anthropic","name":"Black-box RNA sequence modeling and design","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blackjack-basic-strategy","name":"Blackjack Basic Strategy","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blend","name":"BLEnD","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Apertus 70B. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 66.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blenderbench-refresh","name":"BlenderBench (Refresh)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on refresh.dev: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blenderworld","name":"BlenderWorld","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on blenderworld.github.io: 13 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blicket-test-causal-reasoning","name":"Blicket Test Causal Reasoning","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-30B-A3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-30B-A3B (#146)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on app.primeintellect.ai: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blimp","name":"BLiMP","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on raw.githubusercontent.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blimp-nl","name":"BLiMP-NL","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (257 entries)."],["scores.models","pass","21 models we track, plus 236 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Olmo 3.1 32B Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is gpt-oss-20b (#143)."],["scores.headroom","pass","The best is 49.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on EuroEval: 24 rows for models we track and 236 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blind-spot","name":"Blind Spot (Carlos Toledo)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blind-spots-bench","name":"Blind-Spots-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blink","name":"BLINK","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (38 entries), launch posts (19 scores) and system cards (24 scores)."],["scores.models","pass","41 models we track, plus 27 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K2.5 (#76)."],["scores.headroom","pass","The best is 81.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on eval.ai: 11 rows for models we track and 27 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blobfish-litigation-bench","name":"Blobfish Litigation Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blocker-deception-openai","name":"Blocker Deception","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (8 scores)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bluebench-ibm","name":"BlueBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","13 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Granite 3.3 8B Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 13 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blueprint-bench","name":"Blueprint-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","8 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash. That’s expected for a test replaced by Blueprint-Bench 2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on andonlabs.com: 10 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blueprint-bench-2","name":"Blueprint-Bench 2","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (38 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","38 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Carbon."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 58.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on andonlabs.com: 38 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blueprintsymvl","name":"BlueprintSymVL","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bluex","name":"BLUEX","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1,165 entries)."],["scores.models","pass","177 models we track, plus 988 other entries on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for Keye-VL-1.5-8B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 95%, past 85% since March 2025."],["import.importer","pass","We read the board on Open Portuguese LLM Leaderboard: 202 rows for models we track and 988 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"blur","name":"BLUR","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","6 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for DeepSeek-R1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 54%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 8 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bm-bench","name":"BM-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"boardbench-0-1","name":"BoardBench 0.1","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 91.4%, past 85% since June 2026."],["import.importer","pass","We read the board on boardbench.dev: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"boardbench-0-2","name":"BoardBench 0.2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on boardbench.dev: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"boardgamebench-homer-quan","name":"BoardGameBench (Homer Quan)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Muse Glimmer."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Inkling-Small (#68)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bold","name":"BOLD","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 2 years ago, in March 2024, for Gemma 2B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 7B (#211)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"boolq","name":"BoolQ","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries), launch posts (11 scores), system cards (25 scores) and claims Epoch AI collected (61 models)."],["scores.models","pass","79 models we track, plus 23 other entries on its leaderboard."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Gemma 3n E4B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Saturated. The best is 91.9%, past 85% since October 2019."],["import.importer","pass","We read the board on super.gluebenchmark.com: 3 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bornholm-bitext-mining","name":"Bornholm Parallel","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on datawrapper.dwcdn.net: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bot-analysis-arena","name":"Bot Analysis Arena","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aitradingcompetition.com: 12 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"botchat","name":"BotChat","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","10 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for Qwen-14B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on open-compass.github.io: 10 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"boundarybench-policy-under-pressure","name":"BoundaryBench: Policy Under Pressure","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4 Nano. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 Nano (#90)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since March 2026."],["import.importer","pass","We read the board on kaggle.com: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bountybench","name":"BountyBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","1 model we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 50%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bouquet","name":"BOUQuET","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries) and system cards (1 score)."],["scores.models","pass","23 models we track, plus 22 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Glimmer."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 4 31B IT (#101)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 23 rows for models we track and 22 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"brainf-k-coding-benchmark","name":"Brainf**k coding benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2026."],["import.importer","pass","We read the board on kaggle.com: 19 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"break-the-grader","name":"Break the Grader","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"brennan-election-conspiracy-chatbot-audit","name":"Brennan Center election conspiracy chatbot audit","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"brennan-election-image-generation-audit","name":"Brennan Center election image generation audit","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","1 model we track, plus 7 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on datawrapper.dwcdn.net: 1 row for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"brennan-election-image-identification","name":"Brennan Center election image identification","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on brennancenter.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"brevity-benchmark-nvidia","name":"Brevity Benchmark","summary":{"pass":3,"warn":1,"fail":1,"info":5,"text":"3 of 5 checks pass, 1 warning, 1 problem"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","fail","Unverified. We haven’t checked it against a primary source yet."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 82.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"brfauna","name":"BRFauna","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bridge-clinical-text","name":"BRIDGE","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (107 entries)."],["scores.models","pass","48 models we track, plus 59 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen3.8-Flash-Next."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 27B (#59)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 171 rows for models we track and 59 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bridge-llm-bench","name":"Bridge LLM Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 56%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bright-retrieval","name":"BRIGHT","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (113 entries)."],["scores.models","pass","113 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on brightbenchmark.github.io: 113 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bright-pro","name":"BRIGHT-Pro","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","3 models we track, plus 17 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Qwen3-Embedding-0.6B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on leaderboard.mteb.org: 3 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"broken-search-tool-openai","name":"Broken Search Tool","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (14 scores)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"brokenarxiv","name":"BrokenArXiv","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries) and launch posts (1 score)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","4 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 84.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on matharena.ai: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"brokenmath","name":"BrokenMath","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","9 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Grok 4 Fast."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on sycophanticmath.ai: 14 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"brokk-power-ranking","name":"Brokk Power Ranking","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","24 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","6 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on slopcop.com: 47 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"brood-war-bench","name":"Brood War Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","6 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2026."],["import.importer","pass","We read the board on bw.swerdlow.dev: 16 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"broverbs","name":"BRoverbs","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","8 models we track, plus 11 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in October 2024, for Claude 3.5 Haiku. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3.5 Haiku (#175)."],["scores.headroom","info","Saturated. The best is 95%, past 85% since September 2024."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"browsecomp","name":"BrowseComp","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries), launch posts (79 scores) and system cards (35 scores)."],["scores.models","pass","85 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 93.3%, past 85% since February 2026."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"browsecomp-extended-mercor","name":"BrowseComp Extended","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (43 entries)."],["scores.models","pass","42 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","pass","5 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 73%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mercor.com: 42 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"browsecomp-long-context-128k","name":"BrowseComp Long Context 128k","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for GPT-5.1. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.1 (#66)."],["scores.headroom","info","Saturated. The best is 90%, past 85% since August 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"browsecomp-long-context-256k","name":"BrowseComp Long Context 256k","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"browsecomp-plus","name":"BrowseComp-Plus","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries), launch posts (3 scores) and system cards (1 score)."],["scores.models","pass","11 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","pass","The best is 70.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on texttron.github.io: 17 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"browsecomp-vl","name":"BrowseComp-VL","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries) and launch posts (3 scores)."],["scores.models","pass","7 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.7-Plus (#79)."],["scores.headroom","pass","The best is 59%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 8 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"browsecomp-zh","name":"BrowseComp-ZH","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries), launch posts (19 scores) and system cards (8 scores)."],["scores.models","pass","29 models we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Seed1.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5 397B-A17B (#78)."],["scores.headroom","pass","The best is 82.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 11 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"browser-use-linkedin","name":"Browser Use LinkedIn","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","9 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on hub.harborframework.com: 21 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"browserart","name":"BrowserART","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for o1-mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 42 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"browserinject-v1","name":"BrowserInject v1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"browsesafe-plus-plus","name":"BrowseSafe++","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","15 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 91.5%, past 85% since February 2026."],["import.importer","pass","We read the board on browsesafe.perplexity.ai: 15 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"browsesafe-adapt","name":"BrowseSafe-Adapt","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.5 Flash-Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on browsesafe.perplexity.ai: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"browsesafe-bench","name":"BrowseSafe-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","6 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 13 months ago, in October 2025, for gpt-oss-safeguard-120b. That’s expected for a test replaced by BrowseSafe++."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 86.3%, past 85% since August 2025."],["import.importer","pass","We read the board on huggingface.co: 20 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"browsing-prompt-injections-openai","name":"Browsing prompt injections","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 99%, past 85% since August 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"brumo-2025","name":"BRUMO 2025","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","42 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for QED-Nano. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since March 2025."],["import.importer","pass","We read the board on MathArena: 45 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bsnlp-2019-shared-task","name":"BSNLP 2019 Shared Task","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"btf-2","name":"BTF-2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4. That’s expected for a test replaced by BTF-3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a Brier score, with no fixed ceiling."],["import.importer","pass","We read the board on evals.futuresearch.ai: 3 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"btf-3","name":"BTF-3","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on evals.futuresearch.ai: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bu-bench-v2","name":"BU Bench V2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","29 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 40 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bu-bench-v2-1","name":"BU Bench V2.1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on browser-use.com: 39 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bucc-2018-bitext-mining-shared-task","name":"BUCC 2018 bitext mining shared task","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bug-in-the-code-stack","name":"Bug in the Code Stack","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","9 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","pass","The best is 82.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on hamming.ai: 9 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"build-a-chip-bench","name":"Build-A-Chip-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 16.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on chipagents.ai: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"buildarena","name":"BuildArena","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","6 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for DeepSeek-V3.1. That’s expected for a test replaced by BuildArena 2.0 Construction Challenge."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 45.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on build-arena.github.io: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"buildarena-2-0","name":"BuildArena 2.0 Construction Challenge","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries)."],["scores.models","pass","40 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on build-arena.github.io: 40 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"builderbench","name":"BuilderBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 98%, past 85% since May 2026."],["import.importer","pass","We read the board on rajghugare19.github.io: 5 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"builtbench","name":"BuiltBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (67 entries)."],["scores.models","pass","2 models we track, plus 65 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Qwen3-Embedding-4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 2 rows for models we track and 65 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bullshitbench","name":"BullshitBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (113 entries)."],["scores.models","pass","108 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 96.4%, past 85% since October 2025."],["import.importer","pass","We read the board on petergpt.github.io: 195 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bullshitbench-2","name":"BullshitBench V2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (143 entries)."],["scores.models","pass","137 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 95%, past 85% since November 2025."],["import.importer","pass","We read the board on petergpt.github.io: 255 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bun-server-bench","name":"bun-server-bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"burger","name":"BURGER","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for π0.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on siyingzhao.github.io: 36 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"burst","name":"BURST","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on openaccess.thecvf.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"buseybench-svg","name":"BuseyBench SVG Test","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (308 entries)."],["scores.models","pass","261 models we track, plus 47 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on buseybench.com: 282 rows for models we track and 47 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"business-evidence-judgment-v4","name":"Business Evidence Judgment v4","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 Mini (#74)."],["scores.headroom","info","Saturated. The best is 90.7%, past 85% since August 2025."],["import.importer","pass","We read the board on kaggle.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"business-operation-moonshot","name":"Business Operation (Moonshot)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (2 entries) and launch posts (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3.1 Pro (#17)."],["scores.headroom","pass","The best is 37.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kimi.com: 2 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"butter-bench","name":"Butter-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 37%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"buttonbench","name":"ButtonBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries)."],["scores.models","pass","33 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 41 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"buyout-game","name":"Buyout Game","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","32 models we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 36 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"buzzbench","name":"BuzzBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","20 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for GPT-4.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#104)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eqbench.com: 20 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bva-case-citation","name":"BVA Case Citation Dataset","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bvb","name":"BVB","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (50 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","50 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on yoloytang.me: 69 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"c-eval","name":"C-Eval","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries), launch posts (34 scores) and system cards (26 scores)."],["scores.models","pass","49 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Saturated. The best is 93.7%, past 85% since December 2024."],["import.importer","pass","We read the board on github.com: 6 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"c2hlsc","name":"C2HLSC","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for Claude 3.5 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","pass","The best is 74%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"c3","name":"C3","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries), launch posts (1 score) and system cards (5 scores)."],["scores.models","pass","6 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for GLM-5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GLM-5 (#89)."],["scores.headroom","info","Saturated. The best is 97.8%, past 85% since August 2025."],["import.importer","pass","We read the board on arxiv.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"c3-bench","name":"C3-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (50 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","31 models we track, plus 20 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for DeepSeek-V3.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 63.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on yupeijei1997.github.io: 37 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cabbq","name":"CaBBQ","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (34 entries)."],["scores.models","pass","11 models we track, plus 23 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Gemma 3 12B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-8B (#203)."],["scores.headroom","pass","The best is 81%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 11 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cabreu","name":"caBreu","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cacae-2025","name":"CACAE 2025","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cad-arena","name":"CAD Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","12 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on normalfactory.com: 12 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cad-spec","name":"cad-spec","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","14 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3-Coder-Next."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-235B-A22B-Instruct (#133)."],["scores.headroom","pass","The best is 78%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 14 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cadclamp-0-1","name":"CADClamp 0.1","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cadclamp-0-2","name":"CADClamp 0.2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on christiannhill.github.io: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cadec","name":"CADEC","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cadec-2","name":"CADECv2","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cadeval","name":"CadEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","13 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for o3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 74%, under 85%, so it still separates models."],["import.importer","pass","We read the board on willpatrick.xyz: 13 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cadexpert","name":"CADExpert","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","3 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5 mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 mini (#92)."],["scores.headroom","pass","The best is 35.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cadgenbench","name":"CADGenBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1,033 entries) and system cards (2 scores)."],["scores.models","pass","13 models we track, plus 1,022 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 45.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 11 rows for models we track and 1,022 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cadmium-fusion360","name":"CADmium Fusion 360 Reconstruction","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cadprompt","name":"CADPrompt","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cadtestbench","name":"CADTestBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 81%, under 85%, so it still separates models."],["import.importer","pass","We read the board on dimitrismallis.github.io: 28 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cagui","name":"CAGUI","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Qwen2.5-VL-7B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 65.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cais-risk-index","name":"CAIS Risk Index","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (69 entries)."],["scores.models","pass","69 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","8 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on dashboard.safe.ai: 69 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cais-text-capabilities-index","name":"CAIS Text Capabilities Index","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (70 entries)."],["scores.models","pass","70 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","8 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on dashboard.safe.ai: 70 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cais-vision-capabilities-index","name":"CAIS Vision Capabilities Index","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (55 entries)."],["scores.models","pass","55 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on dashboard.safe.ai: 55 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"caliper-bench-v3","name":"Caliper Bench v3","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (887 entries)."],["scores.models","pass","71 models we track, plus 816 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on caliperbench.com: 87 rows for models we track and 816 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"callhome-american-english-speech","name":"CALLHOME American English Speech","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"callhome-diarization","name":"CALLHOME diarization","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"calm","name":"CaLM","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","8 models we track, plus 20 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for Qwen-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 2 (#192)."],["scores.headroom","pass","The best is 45.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on opencausalab.github.io: 8 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"caltech-101","name":"Caltech-101","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"caltech-ucsd-birds-200","name":"Caltech-UCSD Birds 200","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"calvin","name":"CALVIN","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on calvin.cs.uni-freiburg.de: 17 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"camel-cultural-bias","name":"CAMeL","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (22 entries)."],["scores.models","pass","22 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on aclanthology.org: 22 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"can-ai-close-the-books-fp-a-reasoning","name":"Can AI close the books? FP&A reasoning","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries)."],["scores.models","pass","38 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since June 2025."],["import.importer","pass","We read the board on kaggle.com: 38 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"can-evaluation-miss-a-useful-reframing","name":"Can evaluation miss a useful reframing?","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"can-i-play-it","name":"Can I Play It?","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"canitedit","name":"CanItEdit","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for Seed Diffusion Preview. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 54.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cap-x-cap-bench","name":"CaP-X (CaP-Bench)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","10 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 32.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on capgym.github.io: 10 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"capa-feature-identification","name":"capa Feature Identification","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"capbench-english","name":"CAPBENCH-English","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on uspolicyagendas.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"capquiz","name":"CapQuiz","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"capr-am","name":"CAPR-am","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","12 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for Gemma 3 270M. That’s expected for a test replaced by CAPR-ja."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 41.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 12 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"capr-ar","name":"CAPR-ar","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","12 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for Gemma 3 270M. That’s expected for a test replaced by CAPR-am."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 70.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 12 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"capr-ja","name":"CAPR-ja","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","12 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for Gemma 3 270M. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","info","Saturated. The best is 93%, past 85% since October 2024."],["import.importer","pass","We read the board on github.com: 12 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"captionsmith","name":"CaptionSmith","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen3.6 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.6 27B (#88)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ctf-collegiate-openai","name":"Capture the Flag (Collegiate)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (13 scores)."],["scores.models","pass","13 models we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5 mini. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 68%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ctf-high-school-openai","name":"Capture the Flag (High School)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (9 scores)."],["scores.models","pass","9 models we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for ChatGPT agent. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 82%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ctf-internal-openai","name":"Capture the Flag (Internal)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (8 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from August 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Saturated. The best is 97.1%, past 85% since April 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ctf-professional-openai","name":"Capture the Flag (Professional)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (21 scores)."],["scores.models","pass","21 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for GPT-5.5 Instant. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 96.3%, past 85% since December 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"car-bench","name":"CAR-bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 70%, under 85%, so it still separates models."],["import.importer","pass","We read the board on car-bench.github.io: 22 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"carebench","name":"CAREBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on joinhandshake.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"caremanage-bench","name":"CareManage-Bench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on kaggle.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"carena-hard-auto","name":"Carena-Hard-Auto","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"careqa","name":"CareQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","4 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for Qwen2-7B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is phi-3-medium 14B (#194)."],["scores.headroom","pass","The best is 76.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"carla-env-sinatras","name":"CARLA Driving Tasks (sinatras)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cascadebench","name":"CascadeBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","7 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen3.6 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 41.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 18 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"case-bench","name":"CASE-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","11 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","info","Saturated. The best is 90.1%, past 85% since April 2024."],["import.importer","pass","We read the board on arxiv.org: 30 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"casehold","name":"CaseHOLD","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-caselaw-v2","name":"CaseLaw v2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (53 entries)."],["scores.models","pass","53 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Grok 4.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 79.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 54 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"castle-cwe","name":"CASTLE","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","7 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for o3-mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"casual-conversations-v2","name":"Casual Conversations v2","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cat-goric-game-benchmark","name":"Cat-Goric Game Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","1 model we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen3.8-Flash-Next."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 12 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"catalanqa","name":"CatalanQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"catalonia-independence-corpus","name":"Catalonia Independence Corpus","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"catarena","name":"CATArena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","7 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for DeepSeek-V3.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on catarena.ai: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"catarena-gomoku","name":"CATArena Gomoku","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","5 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for DeepSeek-V3.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on catarena.ai: 5 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"catarena-oibench-5","name":"CATArena OIBench-5","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for LongCat-Flash-Chat."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on catarena.ai: 5 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"catarena-pommerman","name":"CATArena Pommerman","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on catarena.ai: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"catarena-swe-perf-10","name":"CATArena SWE-Perf-10","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for LongCat-Flash-Chat."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","pass","We read the board on catarena.ai: 5 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"catarena-tetris","name":"CATArena Tetris","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on catarena.ai: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"catarena-holdem","name":"CATArena Texas Holdem","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","5 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for DeepSeek-V3.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on catarena.ai: 5 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"catastrophic-biology-scenario-uplift-trial-anthropic","name":"Catastrophic biology scenario uplift trial (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Mythos Preview. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"threecb","name":"Catastrophic Cyber Capabilities Benchmark (3CB)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","10 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for o1-mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","pass","The best is 69%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"catcola","name":"CatCoLA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cathedralbench","name":"CathedralBench","summary":{"pass":4,"warn":0,"fail":1,"info":5,"text":"4 of 5 checks pass, 1 problem"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","fail","Unverified. We haven’t checked it against a primary source yet."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.7 (#37)."],["scores.headroom","pass","The best is 29%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cattle-trade","name":"Cattle Trade","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"caugym","name":"CauGym","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"causalreasoningenv","name":"CausalReasoningEnv","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"causalvqa","name":"CausalVQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries) and system cards (2 scores)."],["scores.models","pass","6 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Cosmos3-Edge."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 70%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cb","name":"CB","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries) and launch posts (1 score)."],["scores.models","pass","4 models we track, plus 23 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in May 2023, for PaLM 2-L. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is PaLM 2-L (#198)."],["scores.headroom","info","Saturated. The best is 96.8%, past 85% since October 2019."],["import.importer","pass","We read the board on super.gluebenchmark.com: 3 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cbrn-conversational-coverage-eval-google","name":"CBRN Conversational Evals","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cbrn-red-teaming-google","name":"CBRN expert red teaming","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cbrn-open-ended-questions-google","name":"CBRN open-ended questions","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Gemini 3 Pro. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cbrn-science-eval-google","name":"CBRN Science Eval","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cbt-bench-level-2","name":"CBT-Bench Level II","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on Slingshot AI: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cc-bench","name":"CC-Bench","summary":{"pass":6,"warn":3,"fail":0,"info":3,"text":"6 of 9 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for DeepSeek-V3.1-Terminus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 4 (#113)."],["scores.headroom","pass","The best is 41.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 4 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cc-bench-v2-backend-zai","name":"CC-Bench-V2 Backend","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GLM-5V-Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","pass","The best is 26.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 3 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cc-bench-v2-frontend-zai","name":"CC-Bench-V2 Frontend","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GLM-5V-Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","pass","The best is 75.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 2 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cc-bench-v2-repo-exploration-zai","name":"CC-Bench-V2 Large Repo Exploration","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GLM-5V-Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","pass","The best is 72.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 3 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cc-bench-v2-chained-tasks-zai","name":"CC-Bench-V2 Multi-Step Chained Tasks","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for GLM-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","pass","The best is 61.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 2 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cc-ocr","name":"CC-OCR","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries), launch posts (14 scores) and system cards (3 scores)."],["scores.models","pass","17 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Qwen3.6 27B. That’s expected for a test replaced by CC-OCR-Bench-V2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.6 Plus (#75)."],["scores.headroom","pass","The best is 83.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cc-ocr-bench-v2","name":"CC-OCR-Bench-V2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries) and launch posts (1 score)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 79.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 16 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ccbench-mmbench","name":"CCBench (MMBench Chinese Culture)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (223 entries)."],["scores.models","pass","152 models we track, plus 71 other entries on its leaderboard."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Granite Vision 3.3 2B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","info","Saturated. The best is 86.9%, past 85% since December 2024."],["import.importer","pass","We read the board on rank.opencompass.org.cn: 157 rows for models we track and 71 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ccpm","name":"CCPM","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for DeepSeek-V3. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","info","Saturated. The best is 92%, past 85% since December 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cdvqa","name":"CDVQA","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cefr-adherence-benchmark","name":"CEFR Adherence Benchmark","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on kaggle.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cekura-agent","name":"Cekura Agent Performance","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","7 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Grok Voice 3.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 81.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.cekura.ai: 7 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cekura-bench","name":"Cekura Bench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on benchmarks.cekura.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cekura-production","name":"Cekura Production Voice Agents","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on benchmarks.cekura.ai: 16 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cekura-real","name":"Cekura Real Human Calls","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","7 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Grok Voice 3.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 42.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.cekura.ai: 7 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cekura-stt","name":"Cekura Speech to Text","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","3 models we track, plus 20 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for GPT-Realtime-Whisper."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on benchmarks.cekura.ai: 3 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cekura-tts","name":"Cekura Text to Speech","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","6 models we track, plus 23 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Fish Audio Drama 3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on benchmarks.cekura.ai: 6 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cekura-turn","name":"Cekura Turn Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","8 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Grok Voice 3.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 84.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.cekura.ai: 8 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cellcaution-reasoning-about-cell-composition","name":"CellCaution: Reasoning About Cell Composition","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 31B IT."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 4 31B IT (#101)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cello","name":"CELLO","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (34 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 34 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Hunyuan-T1."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 76.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 34 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"propaganda-eval-nous","name":"Censorship & Propaganda Eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cents-matter","name":"Cents Matter","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.7 Flash (#26)."],["scores.headroom","info","Saturated. The best is 97.5%, past 85% since August 2025."],["import.importer","pass","We read the board on kaggle.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ceo-bench","name":"CEO-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","19 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on ceobench.com: 19 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cequest","name":"CEQuest","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 95.5%, past 85% since February 2026."],["import.importer","pass","We read the board on ai.onestruction.com: 20 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"certified-sommelier-theory","name":"Certified Sommelier (theory knowledge)","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 3 years ago, in March 2023, for GPT-4. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Saturated. The best is 86%, past 85% since March 2023."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cf-div2-stepfun","name":"CF-Div2-Stepfun","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries) and system cards (1 score)."],["scores.models","pass","8 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Step 3.5 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Saturated. The best is 86.1%, past 85% since February 2026."],["import.importer","pass","We read the board on huggingface.co: 8 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cfbench","name":"CFBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (51 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","22 models we track, plus 31 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Baichuan-M2-32B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-preview (#149)."],["scores.headroom","pass","The best is 81%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 22 rows for models we track and 31 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cfdcodebench","name":"CFDCodeBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash. That’s expected for a test replaced by CFDCodeBench V2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.5 Flash (#112)."],["scores.headroom","pass","The best is 14.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on nithinsomu.github.io: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cfdcodebench-2","name":"CFDCodeBench V2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 75%, under 85%, so it still separates models."],["import.importer","pass","We read the board on nithinsomu.github.io: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cfdquery","name":"CFDQuery","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","10 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Saturated. The best is 91.1%, past 85% since January 2025."],["import.importer","pass","We read the board on nithinsomu.github.io: 10 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cflue","name":"CFLUE","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","8 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for Qwen-1_8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-70B (#210)."],["scores.headroom","pass","The best is 53.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cflue-application","name":"CFLUE Application","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","4 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in October 2023, for ChatGLM3-6B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-70B (#210)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cflue-knowledge","name":"CFLUE Knowledge","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","8 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for Qwen-1_8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-70B (#210)."],["scores.headroom","pass","The best is 53.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cg-bench","name":"CG-Bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","9 models we track, plus 16 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Seed2.0 Lite (0428)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 65.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cg-bench.github.io: 6 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chabsa","name":"chABSA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cot-comprehensibility-google","name":"Chain-of-thought comprehensibility","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Saturated. The best is 99.98%, past 85% since November 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cot-faithfulness-clues-anthropic","name":"Chain-of-thought faithfulness with clues (Anthropic)","summary":{"pass":6,"warn":2,"fail":0,"info":4,"text":"6 of 8 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on anthropic.com: 1 row for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cot-informativeness-google","name":"Chain-of-thought informativeness","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Saturated. The best is 99.8%, past 85% since November 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chainworld","name":"ChainWorld","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 31%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bio-red-teamer-challenging-prompts-openai","name":"Challenging prompts from red teamers with biosafety-relevant PhDs","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5 mini. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 93.6%, past 85% since July 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"challenging-red-teaming-eval-1-openai","name":"Challenging Red Teaming Evaluation 1 (created for o3-mini)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 21 months ago, in February 2025, for GPT-4.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-4.5 (#136)."],["scores.headroom","pass","The best is 51%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"challenging-red-teaming-eval-2-openai","name":"Challenging Red Teaming Evaluation 2 (created for deep research)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 21 months ago, in February 2025, for GPT-4.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-4.5 (#136)."],["scores.headroom","pass","The best is 67%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"challenging-refusal-evals-openai","name":"Challenging Refusal Evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (9 scores)."],["scores.models","pass","9 models we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for o3. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o3 (#84)."],["scores.headroom","info","Saturated. The best is 93.4%, past 85% since September 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"challenging-translation-human-eval-zai","name":"Challenging translation human evaluation (Z.ai)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GLM-4.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"changemyview-openai","name":"ChangeMyView","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 21 months ago, in February 2025, for OpenAI deep research. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1 (#118)."],["scores.headroom","info","Saturated. The best is 89.1%, past 85% since December 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"channel-switching-eval","name":"Channel Switching Eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chaos-bench-ocr","name":"CHAOS-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries) and launch posts (1 score)."],["scores.models","pass","3 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-OCR-1.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 29.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"charades-sta","name":"Charades-STA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for TML-Interaction-Small."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 64%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"social-engineering-benchmark-charlemagne","name":"Charlemagne Labs social-engineering benchmark","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"charm-offensive-google","name":"Charm offensive","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in July 2024, for Gemma 2 27B. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-chart-arena","name":"Chart Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","7 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 15 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chart-info-2024","name":"CHART-Info 2024","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chartmimic","name":"ChartMimic","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (17 entries) and system cards (1 score)."],["scores.models","pass","15 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Qwen3-VL-235B-A22B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on chartmimic.github.io: 14 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chartmuseum","name":"ChartMuseum","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries), launch posts (3 scores) and system cards (5 scores)."],["scores.models","pass","25 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","pass","The best is 80.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on chartmuseum-leaderboard.github.io: 21 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 1 pair differs by more than 10 points, the largest for GLM-4.5V (+15 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chartography","name":"Chartography","summary":{"pass":9,"warn":2,"fail":0,"info":1,"text":"9 of 11 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (56 entries), launch posts (16 scores) and system cards (11 scores)."],["scores.models","pass","64 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","8 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 90.7%, past 85% since September 2026."],["import.importer","pass","We read the board on surgehq.ai: 77 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","7 of 10 pairs differ by more than 10 points, the largest for Claude Opus 5 (−55 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chartqa","name":"ChartQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries), launch posts (29 scores) and system cards (23 scores)."],["scores.models","pass","37 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for North Micro Vision Instruct."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3.5 Sonnet (October 2024) (#150)."],["scores.headroom","info","Saturated. The best is 90.9%, past 85% since May 2024."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chartqa-pro","name":"ChartQA Pro","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (21 entries), launch posts (7 scores) and system cards (7 scores)."],["scores.models","pass","15 models we track, plus 14 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Intern-S2-397B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","pass","The best is 72.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 21 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chartx","name":"ChartX","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","5 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in December 2023, for CogAgent."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"charxiv","name":"CharXiv","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (94 entries), launch posts (67 scores) and system cards (35 scores)."],["scores.models","pass","148 models we track, plus 17 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 93.5%, past 85% since December 2025."],["import.importer","pass","We read the board on charxiv.github.io: 78 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 6 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"charxiv-dq","name":"CharXiv (DQ)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (91 entries) and system cards (7 scores)."],["scores.models","pass","73 models we track, plus 24 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","info","Saturated. The best is 95.5%, past 85% since December 2024."],["import.importer","pass","We read the board on github.com: 68 rows for models we track and 24 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"charxiv-extended-mercor","name":"CharXiv Extended","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","38 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 87.1%, past 85% since July 2026."],["import.importer","pass","We read the board on mercor.com: 40 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"charxiv-missing-image-openai","name":"CharXiv Missing Image","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 11 months ago, in December 2025, for GPT-5.2. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"charxiv-missing-images-meta","name":"CharXiv Missing Images","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chat-all-in-one-bench-moonshot","name":"Chat All-in-One Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Saturated. The best is 85.2%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"isd-chatbots-and-the-ballot-box","name":"Chatbots and the Ballot Box","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 89.3%, past 85% since April 2026."],["import.importer","pass","We read the board on isdglobal.org: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"zheng-mof-synthesis-extraction","name":"ChatGPT Chemistry Assistant MOF synthesis extraction","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chatrag-bench","name":"ChatRAG Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries) and system cards (1 score)."],["scores.models","pass","3 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Cohere Command A."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-70B (#182)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 3 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chcatext","name":"ChCatExt","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cheatbench","name":"CheatBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on cheatbench.ai: 10 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"capability-eval-cheating-audit-google","name":"Cheating audit on capability evaluations","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"checkerbench","name":"CheckerBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","pass","The best is 45.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 18 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chem-bio-dynamic-jailbreak-eval-google","name":"Chem / Bio Dynamic Jailbreak Eval","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chemargus","name":"ChemArgus","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 64.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cosmosmind.ai: 75 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chembench","name":"ChemBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (68 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","27 models we track, plus 42 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","pass","The best is 83.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on datasets-server.huggingface.co: 26 rows for models we track and 42 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chemcotbench","name":"ChemCoTBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on howardli1984.github.io that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chemdirt","name":"ChemDIRT","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (21 entries)."],["scores.models","pass","8 models we track, plus 13 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.5 Flash-Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 5 (#35)."],["scores.headroom","pass","The best is 48.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chemicaldiagramqa","name":"ChemicalDiagramQA","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 2 years ago, in May 2024, for Gemini 1.5 Pro. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 1.5 Pro (#174)."],["scores.headroom","pass","The best is 69.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cheminsight","name":"ChemInsight","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","7 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chempro","name":"ChemPro","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","3 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-14B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 38.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chemprot","name":"ChemProt","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on biocreative.bioinformatics.udel.edu: 7 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chess-engine-benchmark-steve-maughan","name":"Chess Engine Benchmark (Steve Maughan)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","5 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chess-llm-benchmark-ryan-brew","name":"Chess LLM Benchmark (Ryan Brew)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","18 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 4 31B IT (#101)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 18 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chess-notation-eval","name":"Chess Notation Evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chess-puzzles","name":"Chess Puzzles","summary":{"pass":8,"warn":1,"fail":0,"info":1,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from Epoch AI’s runs (95 models)."],["scores.models","pass","95 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 79%, under 85%, so it still separates models."],["import.importer","warn","The record links a board on epoch.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chessbench","name":"ChessBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","1 model we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for GPT-3.5 Turbo Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 66.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 10 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chessbench-madebylupu","name":"ChessBench (MadeByLupu)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on chessbench.org: 32 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chest-imagenome","name":"Chest ImaGenome","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chestx-ray14","name":"ChestX-ray14","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on Code with Papers: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chexpert","name":"CheXpert","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (199 entries)."],["scores.models","pass","199 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on stanfordmlgroup.github.io: 199 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chi-bench","name":"CHI-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","23 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 54.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on actava.ai: 41 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chid","name":"CHID","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for DeepSeek LLM 67B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 72.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"child-safety-xai","name":"Child safety (xAI)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"computer-use-child-safety-action-eval-google","name":"Child Safety Action Testing","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"child-safety-evaluations-anthropic","name":"Child safety evaluations (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"csae-safety-eval-google","name":"Child Sexual Abuse and Exploitation Safety","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"childmandarin","name":"ChildMandarin","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","1 model we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for Qwen2-Audio-7B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chime-6-track-2","name":"CHiME-6 Track 2: Unsegmented Speech Recognition","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on chimechallenge.github.io: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chinabench","name":"ChinaBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 84.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on chinabench.vercel.app: 8 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ccpa-2025","name":"Chinese Classical Poetry Appreciation 2025","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on cips-cl.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ccpa-2026","name":"Chinese Classical Poetry Appreciation 2026","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chinese-complex-if-bytedance","name":"Chinese Complex Instruction Following (ByteDance Seed in-house)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Seed2.0 Pro. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 75.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cedcc","name":"Chinese Essay Discourse Coherence Corpus","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cefe-2023","name":"Chinese Essay Fluency Evaluation 2023","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in March 2023, for ChatGLM-6B. That’s expected for a test replaced by Chinese Essay Fluency Evaluation 2024."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cefe-2024","name":"Chinese Essay Fluency Evaluation 2024","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cerru-2024","name":"Chinese Essay Rhetoric Recognition and Understanding 2024","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cerre-2025","name":"Chinese Essay Rhetoric Recognition Evaluation 2025","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clge","name":"Chinese Language Generation Evaluation (CLGE)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clcc-flageval","name":"Chinese Linguistics and Cognition Challenge","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chinese-literature-ner-re","name":"Chinese Literature Named Entity Recognition and Relation Extraction","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cmrc-2017","name":"Chinese Machine Reading Comprehension 2017","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cmrc-2019","name":"Chinese Machine Reading Comprehension 2019","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on ymcui.com: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chinese-simpleqa","name":"Chinese SimpleQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries), launch posts (10 scores) and system cards (10 scores)."],["scores.models","pass","40 models we track, plus 14 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Ling-2.6-1T."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V4-Pro (#70)."],["scores.headroom","info","Saturated. The best is 86%, past 85% since February 2026."],["import.importer","pass","We read the board on livingfuturelab.github.io: 27 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chinese-student-english-essay","name":"Chinese Student English Essay","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chinese-tool-use-benchmark","name":"Chinese Tool-Use Benchmark","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (3 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 2 years ago, in February 2024, for Qwen1.5-72B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen-14B (#214)."],["scores.headroom","info","Saturated. The best is 98%, past 85% since September 2023."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chinese-wplc","name":"Chinese WPLC","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chineseqa","name":"ChineseQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in November 2023, for DeepSeek LLM 67B. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 85.1%, past 85% since November 2023."],["import.importer","pass","We read the board on raw.githubusercontent.com: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chipbench","name":"ChipBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 30.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chipbench-macro-placement","name":"ChiPBench (chip placement)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chipgptv","name":"ChipGPTV","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for GPT-4V."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 71.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chjgbench","name":"ChJGBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","6 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"choubun","name":"ChouBun","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Llama 3-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-8B (#206)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 3 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chrome-commit-scanning-google","name":"Chrome Production Commit Scanning (Google)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries) and launch posts (1 score)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.5 Flash Cyber."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 72%, under 85%, so it still separates models."],["import.importer","pass","We read the board on deepmind.google: 3 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chronomagic-bench","name":"ChronoMagic-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 14 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chronos-benchmark-ii","name":"Chronos Benchmark II","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 20 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cibench","name":"CIBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","8 models we track, plus 16 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","pass","The best is 75.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on open-compass.github.io: 8 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cifar-10","name":"CIFAR-10","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cifar-100","name":"CIFAR-100","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cimemories","name":"CIMemories","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Glimmer."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark (#54)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cinerecon","name":"CineRecon","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on cinerecon.github.io: 13 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ciphergrid","name":"CIPHERGRID","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"circuitrubric","name":"CircuitRubric","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (35 entries)."],["scores.models","pass","28 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for GLM-5.2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 96%, past 85% since April 2025."],["import.importer","pass","We read the board on github.com: 178 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"circuitsense","name":"CircuitSense","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GLM-4.5V."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 68.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on circuitsense-benchmark.github.io: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cirr","name":"CIRR","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (52 entries)."],["scores.models","pass","1 model we track, plus 51 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 22%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MTEB: 1 row for models we track and 51 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"citebench","name":"CiteBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"citevqa","name":"CiteVQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","16 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 76%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 16 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cityscapes","name":"Cityscapes","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (278 entries)."],["scores.models","pass","278 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on cityscapes-dataset.com: 278 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"civbench","name":"CivBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","21 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on vox-deorum.github.io: 38 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"civbench-civilization-vi","name":"CivBench (Civilization VI)","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"civil-bench-tongji","name":"Civil-Bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","6 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for DeepSeek-R1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-R1 (#122)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on specialist.opencompass.org.cn: 6 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"civilcomments-wilds","name":"CivilComments-WILDS","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on wilds.stanford.edu: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cl-bench","name":"CL-bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries), launch posts (6 scores) and system cards (2 scores)."],["scores.models","pass","29 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Tencent Hy3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 27.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on clbench.com: 32 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cl-bench-life","name":"CL-bench Life","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries) and launch posts (2 scores)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from July 2026, for Tencent Hy3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 22.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on clbench.com: 29 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cladbench-v1","name":"CladBench v1","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cladder","name":"CLadder","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","7 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for GPT-4 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","pass","The best is 70.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-claimbench","name":"ClaimBench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","28 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on vals.ai: 28 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clara-med-simplification","name":"CLARA-MeD Spanish Medical Text Simplification","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clash-of-ais-season-1","name":"Clash of AIs Season 1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","16 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on clashofais.com: 18 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clasharness","name":"CLASHarness","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 28.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 35 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"classeval","name":"ClassEval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on fudanselab-classeval.github.io: 12 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ched","name":"Classical Chinese Historical Event Detection","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 15 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"claw-eval","name":"Claw-Eval","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries), launch posts (13 scores) and system cards (1 score)."],["scores.models","pass","34 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 81.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on claw-eval.github.io: 24 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 3 pairs differs by more than 10 points, the largest for MiMo-V2-Omni (−18 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"claw-eval-pass-3","name":"Claw-Eval Pass^3","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries), launch posts (12 scores) and system cards (2 scores)."],["scores.models","pass","32 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for LongCat-2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 78.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on claw-eval.github.io: 24 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 5 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"claw-eval-live","name":"Claw-Eval-Live","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","12 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Qwen 3.6 Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 66.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on claw-eval-live.github.io: 12 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"claw-eval-mm","name":"Claw-Eval-MM","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (11 entries), launch posts (6 scores) and system cards (1 score)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 80.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on claw-eval.github.io: 11 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"claw-swe-bench","name":"Claw-SWE-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (35 entries)."],["scores.models","pass","16 models we track, plus 19 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for MiMo-V2.6 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 79.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on claw-swe-bench.github.io: 36 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clawbench","name":"ClawBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini. That’s expected for a test replaced by ClawBench V2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 61.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clawbench-v2","name":"ClawBench V2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 44.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on claw-bench.com: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clawprobench","name":"ClawProBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (65 entries)."],["scores.models","pass","50 models we track, plus 15 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","pass","We read the board on suyoumo.github.io: 56 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clbench-v","name":"CLBench-V","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Doubao Seed 2.0 Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 28.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clear-clinical-information-extraction","name":"CLEAR (Clinical Information Extraction)","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clear-french-medical-simplification","name":"CLEAR French Medical Text Simplification","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","19 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 19 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clearharm-far-ai","name":"ClearHarm","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-simpletext-2025-simplify-scientific-text","name":"CLEF 2025 SimpleText Task 1","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","5 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for GPT-4.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4.1 (#138)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ceur-ws.org: 8 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-simpletext-2025-identify-avoid-hallucination","name":"CLEF 2025 SimpleText Task 2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ceur-ws.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-simpletext-2026-simplify-scientific-text","name":"CLEF 2026 SimpleText Task 1","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ceur-ws.org: 1 row for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-simpletext-2026-identify-avoid-hallucination","name":"CLEF 2026 SimpleText Task 2","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ceur-ws.org: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-tar-2019-task-2","name":"CLEF eHealth 2019 Technology-Assisted Reviews Task 2","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-hipe-2022","name":"CLEF HIPE 2022","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on hipe-eval.github.io that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-hipe-2026","name":"CLEF HIPE 2026","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on hipe-eval.github.io: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-joker-2023-pun-detection","name":"CLEF JOKER 2023 Task 1: Pun Detection","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","1 model we track, plus 7 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on ceur-ws.org: 3 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-joker-2023-pun-location-and-interpretation","name":"CLEF JOKER 2023 Task 2: Pun Location and Interpretation","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on ceur-ws.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-joker-2023-pun-translation","name":"CLEF JOKER 2023 Task 3: Pun Translation","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","1 model we track, plus 8 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ceur-ws.org: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-joker-2024-humor-aware-retrieval","name":"CLEF JOKER 2024 Task 1: Humour-Aware Information Retrieval","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ceur-ws.org: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-joker-2024-humor-classification","name":"CLEF JOKER 2024 Task 2: Humour Classification According to Genre and Technique","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (52 entries)."],["scores.models","pass","52 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on ceur-ws.org: 52 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-joker-2024-pun-translation","name":"CLEF JOKER 2024 Task 3: Pun Translation","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries)."],["scores.models","pass","17 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ceur-ws.org: 17 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-joker-2025-humor-aware-retrieval","name":"CLEF JOKER 2025 Task 1: Humour-Aware Information Retrieval","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ceur-ws.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-joker-2025-pun-translation","name":"CLEF JOKER 2025 Task 2: Wordplay Translation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (44 entries)."],["scores.models","pass","44 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on ceur-ws.org: 44 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-joker-2025-onomastic-wordplay-translation","name":"CLEF JOKER 2025 Task 3: Onomastic Wordplay Translation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","19 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on ceur-ws.org: 19 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-joker-2026-humor-aware-retrieval","name":"CLEF JOKER 2026 Task 1: Humour-Aware Information Retrieval","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ceur-ws.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-joker-2026-pun-translation","name":"CLEF JOKER 2026 Task 2: Pun Translation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (24 entries)."],["scores.models","pass","24 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on codabench.org: 24 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-joker-2026-onomastic-wordplay-translation","name":"CLEF JOKER 2026 Task 3: Onomastic Wordplay Translation","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","3 models we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 71.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ceur-ws.org: 3 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clef-joker-2026-humor-generation","name":"CLEF JOKER 2026 Task 4: Humour Generation","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","4 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Kimi K2.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on joker-project.com: 5 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clembench","name":"clembench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from July 2026, for Apertus 1.5 8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on clembench.github.io: 9 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cleva","name":"CLEVA","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 3 years ago, in June 2023, for Vicuna-13B-v1.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on crfm.stanford.edu: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cli-bench-one-shot","name":"CLI-Bench One-Shot","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since August 2025."],["import.importer","pass","We read the board on kaggle.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"click-korean","name":"CLIcK","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (4 scores) and system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from August 2026, for K-EXAONE 2.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 90.7%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"click-calibrate-prime","name":"Click Calibrate","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"click-id","name":"CLICK-ID","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (193 entries)."],["scores.models","pass","5 models we track, plus 188 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 65.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MTEB: 5 rows for models we track and 188 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clif-bench-moonshot","name":"CLIF Bench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 52.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"climate-tournament-belief-policy-forecasts","name":"Climate intervention forecasts for beliefs and policy support","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"climate-change-ner","name":"Climate-Change-NER","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 48 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"climate-fever","name":"CLIMATE-FEVER","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (248 entries)."],["scores.models","pass","3 models we track, plus 245 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Qwen3-Embedding-0.6B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MTEB: 3 rows for models we track and 245 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"climatecheck-2025","name":"ClimateCheck 2025","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"climatecheck-2026","name":"ClimateCheck 2026","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clinhallu","name":"ClinHallu","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","9 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 80.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clinical-abbreviation-sense-inventory","name":"Clinical Abbreviation Sense Inventory","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clinical-laboratory-test-reports","name":"Clinical Laboratory Test Reports","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omi-soap-note-safety","name":"Clinical SOAP Note Evaluation: Safety-First Benchmarking","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on omi.health: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clinicare-bench","name":"CliniCARE-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 76.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clip-representation-bias-evaluation","name":"CLIP representation bias evaluation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on proceedings.iclr.cc: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clipping-bench","name":"Clipping Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on usenaive.ai: 19 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clipsep-sound-separation","name":"CLIPSep Sound Separation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clockbench","name":"ClockBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries)."],["scores.models","pass","40 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 87.8%, past 85% since October 2026."],["import.importer","pass","We read the board on clockbench.ai: 40 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clocktower-bench","name":"Clocktower Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on clocktowerbench.com: 11 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clocktower-radio","name":"Clocktower Radio","summary":{"pass":8,"warn":0,"fail":1,"info":3,"text":"8 of 9 checks pass, 1 problem"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","23 models we track."],["scores.recent","pass","The newest is from October 2026, for Mistral Large 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on clocktower-radio.com: 28 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","fail","The read on November 19th, 2026 failed with “403 https://clocktower-radio.com/”. Pages show the rows from November 18th, 2026."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"closed-domain-hallucination-openai","name":"Closed-domain hallucination evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 3 years ago, in March 2023, for GPT-4. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"closed-loop-embedded-firmware","name":"Closed-Loop Evaluation of LLM Agents for Embedded Software Development","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since March 2026."],["import.importer","pass","We read the board on github.com: 28 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clothdojo","name":"ClothDojo","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GR00T N1.7."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 44.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on clothdojo.github.io: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clotho","name":"Clotho","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on SOTA2 Research: 15 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clotho-ovl-rel","name":"Clotho OVL-REL Test Set","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clotho-aqa","name":"Clotho-AQA","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for ERNIE 5.0. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 73.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cloudexp-ce80-10","name":"Cloud Expansion CE80-10","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cloudexp-ceexfil-0","name":"Cloud Expansion CEExfil-0","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cloudexp-rx00","name":"Cloud Expansion RX00","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clue","name":"CLUE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cluener2020","name":"CLUENER2020","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cluewsc","name":"CLUEWSC","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries), launch posts (4 scores) and system cards (7 scores)."],["scores.models","pass","8 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V4-Pro (#70)."],["scores.headroom","info","Saturated. The best is 92.8%, past 85% since May 2024."],["import.importer","pass","We read the board on github.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"clutterbench","name":"ClutterBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on clutterbench.adamholter.chatgpt.site: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cmath","name":"CMATH","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (4 scores) and system cards (3 scores)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V4-Pro (#70)."],["scores.headroom","info","Saturated. The best is 96.7%, past 85% since December 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cmb","name":"CMB","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (31 entries)."],["scores.models","pass","6 models we track, plus 25 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for DeepSeek LLM 67B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 66.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 24 rows for models we track and 25 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cmd-ad-eval","name":"CMD-AD-Eval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cmi-bench","name":"CMI-Bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Gemini 3.1 Flash-Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Flash-Lite (#96)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MuTheoryEval: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cmi-rewardbench","name":"CMI-RewardBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","3 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cmimc-2025","name":"CMIMC 2025","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","33 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for QED-Nano. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Saturated. The best is 95.6%, past 85% since July 2025."],["import.importer","pass","We read the board on MathArena: 36 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cmmlu","name":"CMMLU","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (47 entries), launch posts (16 scores) and system cards (8 scores)."],["scores.models","pass","27 models we track, plus 36 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V4-Pro (#70)."],["scores.headroom","info","Saturated. The best is 90.8%, past 85% since April 2024."],["import.importer","pass","We read the board on github.com: 22 rows for models we track and 36 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cmmmu","name":"CMMMU","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","9 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for InternVL-Chat-ViT-6B-Vicuna-13B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 36.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cmmu","name":"CMMU","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","5 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for Qwen-VL-Plus."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 10 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cmo-2024","name":"CMO 2024","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for DeepSeekMath-V2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 73.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cmo-2025","name":"CMO 2025","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Seed2.0 Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cmrc","name":"CMRC","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 40 other entries on its leaderboard."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for DeepSeek-V3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","pass","The best is 76.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ymcui.com: 40 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cmt-benchmark","name":"CMT-Benchmark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (16 entries) and launch posts (2 scores)."],["scores.models","pass","13 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Tencent Hy3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 50.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 12 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cnfinbench","name":"CNFinBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","10 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on cnfinbench.opencompass.org.cn: 14 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cnml-level-problems-deepseek","name":"CNML-Level Problems (DeepSeek)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 12 months ago, in November 2025, for DeepSeekMath-V2. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cnmo-2024","name":"CNMO 2024","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries), launch posts (5 scores) and system cards (5 scores)."],["scores.models","pass","13 models we track."],["scores.recent","info","The newest came 10 months ago, in January 2026, for STEP3-VL-10B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K2 (#117)."],["scores.headroom","info","Saturated. The best is 88.5%, past 85% since May 2025."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cnn-dailymail-summarization","name":"CNN/DailyMail Summarization","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","6 models we track, plus 36 other entries on its leaderboard."],["scores.recent","warn","The newest came 4 years ago, in July 2022, for BLOOM-176B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on HELM Classic: 6 rows for models we track and 36 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"co-design-bench","name":"Co-Design Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 69.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on codesignbench.github.io: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cobench-anthropic","name":"CoBench (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 55.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cochrane-rct-summarization","name":"Cochrane RCT Summarization","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cochrane-simplification","name":"Cochrane Simplification","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cochrane-auto","name":"Cochrane-auto","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coco","name":"COCO","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (216 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 216 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 78.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cocodataset.org: 216 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coco-captions","name":"COCO Captions","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (76 entries), launch posts (1 score) and system cards (3 scores)."],["scores.models","pass","3 models we track, plus 76 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Gemma 3 12B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on cocodataset.org: 76 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coco-o","name":"COCO-O","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (53 entries)."],["scores.models","pass","53 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 53 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coco-stuff-164k","name":"COCO-Stuff 164K","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on MMSegmentation: 12 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cocoabench","name":"CocoaBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 45.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cocoabench.github.io: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coconot","name":"CoCoNot","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries)."],["scores.models","pass","10 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Llama 3-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 20 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cocoteros","name":"COCOTEROS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"code-arena","name":"Code Arena","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (134 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","131 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Qwen4-Max."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arena.ai: 154 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 5 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"code-interpreter-benchmark","name":"Code Interpreter Benchmark","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for Qwen-1_8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen-14B (#214)."],["scores.headroom","pass","The best is 81.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"code-interpreter-benchmark-general","name":"Code Interpreter Benchmark General","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","2 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in October 2023, for ChatGLM3-6B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen-14B (#214)."],["scores.headroom","pass","The best is 65.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"code-interpreter-benchmark-math","name":"Code Interpreter Benchmark Math","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries) and launch posts (2 scores)."],["scores.models","pass","3 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Qwen1.5-72B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen-14B (#214)."],["scores.headroom","pass","The best is 73.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-code-migration","name":"Code Migration","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (78 entries)."],["scores.models","pass","78 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 69.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 79 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-code-migration-1-1","name":"Code Migration v1.1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries) and system cards (1 score)."],["scores.models","pass","31 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","6 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 86.9%, past 85% since September 2026."],["import.importer","pass","We read the board on vals.ai: 30 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"code-refactoring-openai","name":"Code refactoring tasks (OpenAI)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for GPT-5-Codex. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 51.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"martian-code-review-bench","name":"Code Review Bench (Martian)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","21 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on codereview.withmartian.com: 21 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"code-summary-honesty-anthropic","name":"Code summary honesty (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"code2video-arena","name":"Code2Video Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","7 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 13 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"code2video-benchmark","name":"Code2Video Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 19 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"code2video-aa","name":"Code2Video-AA","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"code2video-compass","name":"Code2Video-Compass","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on code2video-compass.github.io: 15 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"code4scene","name":"Code4Scene","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on simworld-ai-code4scene.static.hf.space: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codecap-bench","name":"CodeCap-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Sol."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codecap-bench.github.io: 16 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codeclash","name":"CodeClash","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on codeclash.ai: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codecmos-accent","name":"CodecMOS-Accent","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (24 entries)."],["scores.models","pass","24 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 24 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codecompass","name":"CodeCompass","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-235B-A22B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-235B-A22B (#131)."],["scores.headroom","pass","The best is 40%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codecontests","name":"CodeContests","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries)."],["scores.models","pass","3 models we track, plus 42 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 81.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on SOTA2 Research: 42 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codeeditorbench","name":"CodeEditorBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","7 models we track, plus 12 other entries on its leaderboard."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Seed-Coder-8B-Instruct. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Saturated. The best is 85.5%, past 85% since June 2023."],["import.importer","pass","We read the board on codeeditorbench.github.io: 19 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codeelo","name":"CodeElo","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (33 entries)."],["scores.models","pass","9 models we track, plus 24 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for QwQ-32B-Preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 24 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codeforces-rating","name":"Codeforces rating","summary":{"pass":3,"warn":1,"fail":1,"info":5,"text":"3 of 5 checks pass, 1 warning, 1 problem"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","fail","Unverified. We haven’t checked it against a primary source yet."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (34 scores) and system cards (33 scores)."],["scores.models","pass","47 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codeinsights","name":"CodeInsights","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codeipi","name":"CodeIPI","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Sonnet 4.6 (#57)."],["scores.headroom","info","Saturated. The best is 91.1%, past 85% since April 2025."],["import.importer","pass","We read the board on github.com: 5 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codepde","name":"CodePDE","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Claude Haiku 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","pass","The best is 38.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Harbor: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codesearchnet-challenge","name":"CodeSearchNet Challenge","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (205 entries)."],["scores.models","pass","4 models we track, plus 201 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MTEB: 4 rows for models we track and 201 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codesimpleqa","name":"CodeSimpleQA","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Seed2.0 Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 58%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codetransocean","name":"CodeTransOcean","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (209 entries)."],["scores.models","pass","4 models we track, plus 205 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MTEB: 4 rows for models we track and 205 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"code-utility-bench-lg","name":"CodeUtilityBench (LG)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 10 months ago, in January 2026, for K-EXAONE. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 71.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codex-of-veritas","name":"codex of veritas","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since March 2026."],["import.importer","pass","We read the board on kaggle.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"codexglue","name":"CodeXGLUE","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on microsoft.github.io: 1 entry at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coding-inhouse-minimax","name":"Coding (MiniMax in-house)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for MiniMax-Text-01. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 90.2%, past 85% since January 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-coding-agent-google","name":"Coding agent prompt injection robustness","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","pass","The best is 83%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coding-deception-openai","name":"Coding Deception","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-coding-experience-moonshot","name":"Coding Experience (Moonshot)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 59.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coding-prompt-injections-openai","name":"Coding prompt injections","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for GPT-5.1-Codex-Max. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since May 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coffeebench-sakana","name":"CoffeeBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cognition-codesearch-eval","name":"Cognition CodeSearch Eval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","6 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Claude Haiku 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on cognition.com: 11 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cognition-golden","name":"cognition-golden","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for o1-mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","pass","The best is 51.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cdn.sanity.io: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterprise-human-evaluation-cohere","name":"Cohere enterprise human evaluation","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries) and launch posts (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Cohere Command A."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#161)."],["scores.headroom","pass","The best is 51.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cohere.com: 2 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cohere-north-internal-evals","name":"Cohere North internal evals","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coin-instructional-video-analysis","name":"COIN instructional video analysis","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on coin-dataset.github.io: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coir","name":"CoIR","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on archersama.github.io: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"colder","name":"ColDeR","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coliee-2024","name":"COLIEE 2024","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coliee-2025","name":"COLIEE 2025","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on coliee.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coliee-2026","name":"COLIEE 2026","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on coliee.org: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"collab-overcooked","name":"Collab-Overcooked","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","5 models we track, plus 8 other entries on its leaderboard."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Claude Sonnet 4. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o4-mini (#95)."],["scores.headroom","info","Saturated. The best is 86.7%, past 85% since May 2025."],["import.importer","pass","We read the board on aclanthology.org: 5 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"collaborative-intelligence-benchmark","name":"Collaborative Intelligence Benchmark","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 82.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on zou-group.github.io: 24 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"collie","name":"COLLIE","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (12 scores) and system cards (5 scores)."],["scores.models","pass","12 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Mistral Medium 3.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 99%, past 85% since August 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"colorbench","name":"ColorBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (32 entries)."],["scores.models","pass","22 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Gemini 2.0 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","pass","The best is 59.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 23 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"combibench","name":"CombiBench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","6 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Seed-Prover 1.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 48%, under 85%, so it still separates models."],["import.importer","pass","We read the board on moonshotai.github.io: 7 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"combined-hhh-evals-anthropic","name":"Combined HHH evals (Anthropic)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries) and system cards (1 score)."],["scores.models","pass","1 model we track, plus 36 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in July 2023, for Claude 2. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 2 (#192)."],["scores.headroom","info","Saturated. The best is 86%, past 85% since July 2023."],["import.importer","pass","We read the board on github.com: 36 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flappy-bird-command-code","name":"Command Code Flappy Bird","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pixel-art-space-shooter-command-code","name":"Command Code Pixel Art Space Shooter","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"commerceagentbench","name":"CommerceAgentBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","39 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on commerce-agent-bench.site.accio.ai: 39 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"common-objects-in-3d-v2","name":"Common Objects in 3D v2","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"common-syntactic-processes-in-text-to-image-generation","name":"Common Syntactic Processes in Text-to-Image Generation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"common-voice","name":"Common Voice","summary":{"pass":5,"warn":1,"fail":0,"info":4,"text":"5 of 6 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (8 scores) and system cards (3 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-Live-Transcribe."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","warn","The record links a board on commonvoice.mozilla.org that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"commonground-elicit","name":"CommonGround Elicit","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Kimi K2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4.1 (#138)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crmus","name":"Commonsense Reasoning and Moral Understanding in Children’s Stories","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"commonsenseqa","name":"CommonSenseQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries), launch posts (5 scores) and system cards (21 scores)."],["scores.models","pass","19 models we track, plus 14 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Saturated. The best is 90.4%, past 85% since May 2023."],["import.importer","pass","We read the board on arxiv.org: 14 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"commonsenseqa-2-0","name":"CommonsenseQA 2.0","summary":{"pass":5,"warn":1,"fail":0,"info":4,"text":"5 of 6 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from claims Epoch AI collected (6 models)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Llama 2-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-3.5 Turbo (#209)."],["scores.headroom","pass","The best is 67.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"community-alignment-personalization-meta","name":"Community Alignment personalization","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"compactionbench","name":"CompactionBench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"company-agent-bench","name":"Company Agent Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 82.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on concept.dev: 24 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"firecrawl-company-enrichment","name":"Company Enrichment API Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on firecrawl.dev: 6 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"companybench-turing","name":"CompanyBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 44.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on bench.turing.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"compass-constrained-optimization","name":"COMPASS","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"compass-academic-leaderboard","name":"Compass Academic Leaderboard","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","15 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Gemma 3 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","pass","The best is 72.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 15 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"compassarena","name":"CompassArena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on rank.opencompass.org.cn: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"compassarena-multimodal","name":"CompassArena Multimodal","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","8 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Qwen3-VL-235B-A22B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on rank.opencompass.org.cn: 8 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"compassbench-1-3","name":"CompassBench v1.3","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"compassrank","name":"CompassRank","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on rank.opencompass.org.cn: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"competitive-programming-code-generation","name":"Competitive Programming Code Generation","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","info","Saturated. The best is 90%, past 85% since September 2026."],["import.importer","pass","We read the board on kaggle.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"compilebench","name":"CompileBench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","info","The newest came 13 months ago, in October 2025, for Claude Haiku 4.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since May 2025."],["import.importer","pass","We read the board on compilebench.com: 26 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finmteb-complaints-clustering","name":"ComplaintsClustering","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"complex-workflow-bytedance","name":"Complex Workflow (ByteDance Seed in-house)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Seed1.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 64.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"complexbench","name":"ComplexBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","12 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Llama 3-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"complexconstraints","name":"ComplexConstraints","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 70.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on surgehq.ai: 16 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"complexfuncbench","name":"ComplexFuncBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries), launch posts (3 scores) and system cards (1 score)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Hunyuan-A13B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4.1 mini (#145)."],["scores.headroom","pass","The best is 61.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"complexfuncbench-audio-openai","name":"ComplexFuncBench Audio (OpenAI)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for gpt-realtime. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 66.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"complexnlu-tencent","name":"ComplexNLU (Tencent)","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (4 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Hunyuan-A13B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 64.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 4 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pasted-instruction-compliance-anthropic","name":"Compliance with instructions planted in pasted text (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"compmat-bench","name":"CompMat-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Saturated. The best is 91.5%, past 85% since July 2026."],["import.importer","pass","We read the board on github.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"composite-bench","name":"Composite-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 74%, under 85%, so it still separates models."],["import.importer","pass","We read the board on research.composite.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"compositional-tasks-bytedance","name":"Compositional Tasks (ByteDance Seed in-house)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Seed2.0 Pro. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 59.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"computeeval-nvidia","name":"ComputeEval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"computer-use-manual-red-teaming-google","name":"Computer use manual red teaming","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"computer-use-prohibited-action-eval-google","name":"Computer use Prohibited Action Evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 99.4%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"computer-use-prompt-injection-anthropic","name":"Computer use prompt injection evaluation (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 13 months ago, in October 2025, for Claude Haiku 4.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Sonnet 4.5 (#81)."],["scores.headroom","pass","The best is 78%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-computer-use-google","name":"Computer use prompt injection robustness","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 92%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"concealment-of-changes-anthropic","name":"Concealment of changes (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 96.9%, past 85% since September 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"conceptedit-bench","name":"ConceptEdit-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","6 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Flash Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"conceptual-reasoning-index","name":"Conceptual Reasoning Index","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (155 entries)."],["scores.models","pass","154 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on conceptualreasoning.ai: 156 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"concurrency-bugs","name":"Concurrency Bugs (tanishgudise)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"conll-2002-named-entity-recognition","name":"CoNLL-2002 Named Entity Recognition","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"connected-robotics-evidence-graph-benchmark","name":"Connected Robotics Evidence Graph Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Gemma 3 12B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 80.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on creg-bench.github.io: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"conref-stance-ita","name":"ConRef-STANCE-ita","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"constory-bench","name":"ConStory-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","18 models we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Ring-1T."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 18 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"construction-ai-leaderboard-onestruction","name":"Construction AI Leaderboard","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on ai.onestruction.com: 20 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"constructions-last-exam","name":"Construction's Last Exam","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","30 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 58.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cle.togal.ai: 30 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"constructions-last-exam-2","name":"Construction's Last Exam v2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Fable 5.1."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","pass","The best is 36.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cle.togal.ai: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"consulting-and-recommendations-benchmark-stepfun","name":"Consulting and Recommendations Benchmark (StepFun)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Step 3.5 Flash. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 70.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"context-aware-asr-openai","name":"Context Aware Automatic Speech Recognition","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for GPT-Live-Transcribe."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 44.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"contextasr-bench","name":"ContextASR-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","7 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Kimi-Audio-7B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 17 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"contextbench","name":"ContextBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","9 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Qwen3.7-Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.7-Max (#49)."],["scores.headroom","pass","The best is 60.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on contextbench.github.io: 14 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"contextual-complex-instructions","name":"Contextual Complex Instruction-Following Retrieval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"contextual-earnings-22","name":"Contextual Earnings-22","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","2 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Qwen3-ASR-1.7B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 6 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"contextual-invoice-ocr-ramp","name":"Contextual Invoice OCR","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on builders.ramp.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"contextual-nuclear-knowledge-openai","name":"Contextual Nuclear Knowledge","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 21 months ago, in February 2025, for GPT-4.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 77%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"contextual-recency-based-instructions","name":"Contextual Recency-Based Instruction-Following Retrieval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"contextual-reranker-technical-documentation","name":"Contextual Reranker Technical Documentation Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-contextual-security-google","name":"Contextual security evaluation","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","pass","The best is 27%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"contextualbench-salesforce","name":"ContextualBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"continual-learning-bench","name":"Continual Learning Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Claude Opus 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on snorkel.ai: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"continual-learning-exam","name":"Continual Learning Exam","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from November 2026, for Doubao Seed Evolving."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 47.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on haoyu-zhao.github.io: 17 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"continualskillbench","name":"ContinualSkillBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Claude Opus 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.7 (#31)."],["scores.headroom","pass","The best is 64.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"contphy","name":"ContPhy","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 67.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"contract-intelligence-harvey","name":"Contract Intelligence","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legalbenchmarks-contract-workflows-applications","name":"Contract Workflows (Applications)","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on legalbenchmarks.ai: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legalbenchmarks-contract-workflows-models","name":"Contract Workflows (Models)","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 48.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on legalbenchmarks.ai: 27 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"contractbench-observation-contracts","name":"ContractBench (Observation Contracts)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","18 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 77.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 18 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"contractnli","name":"ContractNLI","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"contractoros-quantity-takeoff","name":"ContractorOS Quantity Takeoff","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (43 entries)."],["scores.models","pass","43 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 60%, under 85%, so it still separates models."],["import.importer","pass","We read the board on contractoros.build: 97 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"controlarena","name":"ControlArena","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"conversational-swe-task-benchmark-windsurf","name":"Conversational SWE Task Benchmark (Windsurf)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 18 months ago, in May 2025, for SWE-1. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"convfinqa","name":"ConvFinQA","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"convomem","name":"ConvoMem","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cookbench-cookwing-v1","name":"CookBench v1 (CookWing)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 40%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cook.redwi.ng: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cooperate-to-compete-c2c","name":"Cooperate to Compete (C2C)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Gemini 3.1 Flash-Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cooperation-with-user-deception-anthropic","name":"Cooperation with user deception (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cooperative-codenames-bittker","name":"Cooperative Codenames (Max Bittker)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cooperbench","name":"CooperBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 27.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cooperbench.com: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"copa","name":"COPA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 10 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in May 2023, for PaLM 2-L. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is PaLM 2-L (#198)."],["scores.headroom","info","Saturated. The best is 96%, past 85% since May 2023."],["import.importer","pass","We read the board on asgordon.github.io: 10 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"copal-id","name":"COPAL-ID","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","1 model we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Llama 2-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-7B (#238)."],["scores.headroom","pass","The best is 58.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"copilot-arena","name":"Copilot Arena","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Gemini 1.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arena.ai: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"copilot-arena-code-editing","name":"Copilot Arena Code Editing","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Gemini 1.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on arena.ai: 3 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"copious-biodiversity-ner","name":"COPIOUS biodiversity entity recognition","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coqa","name":"CoQA","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","42 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on stanfordnlp.github.io: 42 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coqcat","name":"CoQCat","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coraal","name":"CORAAL","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coral","name":"CORAL","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Qwen2.5-1.5B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coralvqa","name":"CoralVQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in October 2024, for Claude 3.5 Haiku."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Haiku (#175)."],["scores.headroom","pass","The best is 41.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"core-english-web-register-identification","name":"CORE English web register identification","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"core-bench","name":"CORE-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Claude Opus 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 77.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on hal.cs.princeton.edu: 49 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"corebench-conversational-reasoning","name":"COREBench (Conversational Reasoning)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","11 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#114)."],["scores.headroom","pass","The best is 81.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on AHELM: 14 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"corecodebench","name":"CoreCodeBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 44.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"corecognition","name":"CoreCognition","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (229 entries) and system cards (1 score)."],["scores.models","pass","122 models we track, plus 108 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Reka Edge."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 74.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on grow-ai-like-a-child.github.io: 122 rows for models we track and 108 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coronavirus-ace2-cell-entry-screen-openai","name":"Coronavirus-ACE2 Cell-Entry Screen","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (10 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-corp-speak-arena","name":"Corp Speak Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","22 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 24 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"corpbench-work","name":"CorpBench Work","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"corpfin-v2","name":"CorpFin v2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (119 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","119 models we track."],["scores.recent","pass","The newest is from August 2026, for DeepSeek V4 Pro 0813."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 73.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 134 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"corporatebench","name":"CorporateBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 82.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Fireworks AI: 13 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"corpus-for-emotion-detection-in-russian","name":"Corpus for Emotion Detection in Russian","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (203 entries)."],["scores.models","pass","4 models we track, plus 199 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 53%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MTEB: 4 rows for models we track and 199 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"south-african-code-switched-soap-opera-speech","name":"Corpus of Multilingual Code-switched Soap Opera Speech","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on isiXhosa Speech AI: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"corpusqa","name":"CorpusQA","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for MAI-Thinking-1."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 82%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"corpusqa-1m","name":"CorpusQA 1M","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V4-Pro (#70)."],["scores.headroom","pass","The best is 62%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"corr2cause","name":"Corr2Cause","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (29 entries)."],["scores.models","pass","5 models we track, plus 24 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in July 2023, for Llama 2-7B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 2-7B (#238)."],["scores.headroom","info","Saturated. The best is 92.9%, past 85% since May 2020."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 24 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cortex-analyst-multi-turn-snowflake","name":"Cortex Analyst Multi-Turn Question Rewriting","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cortex-retrieval-bench","name":"CortexRetrievalBench","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on micro1.ai: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cose-e","name":"CoSE-E","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on aclanthology.org: 3 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coshe-eval","name":"CoSHE-Eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cosqa","name":"CoSQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (213 entries)."],["scores.models","pass","4 models we track, plus 209 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MTEB: 4 rows for models we track and 209 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cost-english-hindi","name":"CoST English-Hindi","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","2 models we track, plus 14 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for Command R+."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-70B (#182)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cost-per-correct","name":"Cost Per Correct","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","warn","The record links a board on kaggle.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cost-per-correct-v2","name":"Cost Per Correct v2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cost-per-successful-task-arize-fireworks","name":"Cost per Successful Task","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on arize.com: 10 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cot-deception-monitoring-openai","name":"CoT Deception Monitoring","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for o1. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1 (#118)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cot-monitorability-openai","name":"CoT Monitorability","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cot-control","name":"CoT-Control","summary":{"pass":7,"warn":0,"fail":0,"info":3,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (12 scores)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 83.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"counselbench-100","name":"CounselBench-100","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on blobfish.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"counselbench-adv","name":"CounselBench-Adv","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mheval.slingshotai.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"counselbench-eval","name":"CounselBench-Eval","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on Slingshot AI: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"countbench","name":"CountBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries), launch posts (21 scores) and system cards (15 scores)."],["scores.models","pass","28 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for North Micro Vision Instruct."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.6 Plus (#75)."],["scores.headroom","info","Saturated. The best is 97.8%, past 85% since February 2025."],["import.importer","pass","We read the board on teaching-clip-to-count.github.io: 10 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"counterbench","name":"CounterBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","6 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 23 months ago, in December 2024, for DeepSeek-V3. That’s expected for a test replaced by CounterBench (1,200 questions)."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","info","Saturated. The best is 93.5%, past 85% since February 2024."],["import.importer","pass","We read the board on arxiv.org: 24 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"counterbench-1200","name":"CounterBench (1,200 questions)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","6 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 23 months ago, in December 2024, for DeepSeek-V3. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","info","Saturated. The best is 91.8%, past 85% since February 2024."],["import.importer","pass","We read the board on arxiv.org: 24 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"counterfactqa-qwen","name":"CounterFactQA (Qwen)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Qwen3-32B. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 68.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"countqa","name":"CountQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","15 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 82.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"country211","name":"Country211","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (121 entries)."],["scores.models","pass","121 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on OpenCLIP: 121 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coval-stt-wer","name":"Coval Speech-to-Text WER","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","4 models we track, plus 25 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.5 Transcribe Live."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on benchmarks.coval.ai: 4 rows for models we track and 25 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coval-tts-ttfa","name":"Coval Text-to-Speech TTFA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","2 models we track, plus 30 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Grok TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on benchmarks.coval.ai: 2 rows for models we track and 30 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coval-voice-benchmarks","name":"Coval Voice AI Benchmarks","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on benchmarks.coval.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"covebench","name":"CoVEBench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","1 model we track, plus 9 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on nju-link.github.io: 1 row for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coverage-cat-fictional-insurance","name":"Coverage Cat AI Insurance Benchmark (fictional insurance tasks)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since August 2026."],["import.importer","pass","We read the board on coveragecat.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"covert-misbehavior-thinking-machines","name":"Covert misbehavior (Thinking Machines)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"covid-dialogue","name":"COVID-Dialogue","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"covost2","name":"CoVoST2","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (10 scores) and system cards (9 scores)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"covr-r","name":"CoVR-R","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","2 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Qwen2.5-VL-3B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 47.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"coworkbench","name":"CoWorkBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (7 scores) and system cards (1 score)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 76.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on qwen.ai: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cozzo-ai-receptionist-round-4","name":"Cozzo AI Receptionist Benchmark (Round 4)","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on cozzo.io that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cozzo-ai-receptionist-round-5","name":"Cozzo AI Receptionist Benchmark (Round 5)","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on cozzo.io: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cozzo-ai-receptionist-round-6","name":"Cozzo AI Receptionist Benchmark (Round 6)","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on cozzo.io: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cpret","name":"CPRet","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cpu-inference-optimization-and-streaming","name":"CPU Inference Optimization and Streaming","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Sonnet 5.5 (#9)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crab-benchmark","name":"CRAB","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 38%, under 85%, so it still separates models."],["import.importer","pass","We read the board on crab.camel-ai.org: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crag-meta","name":"CRAG","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","2 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for GPT-4 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#172)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crag-mm","name":"CRAG-MM","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crate-bench","name":"Crate Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crazyrouter-ai-trading-arena","name":"Crazyrouter AI Trading Arena","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is -7.45%, under 85%, so it still separates models."],["import.importer","pass","We read the board on crazyrouter.com: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"create-visual-remote-associates-test","name":"CREATE","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 27B (#59)."],["scores.headroom","pass","The best is 42.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 20 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"creation-mmbench","name":"Creation-MMBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","19 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for GPT-4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4.5 (#136)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on open-compass.github.io: 19 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"creative-biology-anthropic","name":"Creative biology","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (7 scores)."],["scores.models","pass","7 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 60.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"creative-biology-uplift-trial-anthropic","name":"Creative Biology Uplift Trial (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"creative-control-llms-as-a-film-crew","name":"Creative Control: LLMs as a Film Crew","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"creative-writing-inhouse-minimax","name":"Creative Writing (MiniMax in-house)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for MiniMax-Text-01. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 81.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"creative-writing-v3","name":"Creative Writing v3","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (144 entries), launch posts (8 scores) and system cards (11 scores)."],["scores.models","pass","131 models we track, plus 23 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eqbench.com: 123 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 2 pairs differs by more than 10 points, the largest for Hermes 4 405B (+13.1)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"creativework-bytedance","name":"CreativeWork","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 42.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on lf3-static.bytednsdoc.com: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"creddata","name":"CredData","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"credit-firewall-benchmark","name":"Credit Firewall Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since October 2025."],["import.importer","pass","We read the board on kaggle.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cref-forecasting","name":"CREF","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on autogluon-fev-bench.hf.space: 26 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crest-arcade-chess","name":"Crest Arcade Chess","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","19 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on glimmerindex.com: 19 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crest-arcade-elemental-armory","name":"Crest Arcade Elemental Armory","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 67.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on glimmerindex.com: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crest-arcade-freecell","name":"Crest Arcade FreeCell","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 63.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on glimmerindex.com: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crest-arcade-hustle-sim","name":"Crest Arcade Hustle Sim","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on glimmerindex.com: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crest-arcade-micro-mouse","name":"Crest Arcade Micro Mouse","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on glimmerindex.com: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crest-arcade-mines","name":"Crest Arcade Mines","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since October 2026."],["import.importer","pass","We read the board on glimmerindex.com: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crest-arcade-nations-royale","name":"Crest Arcade Nations Royale","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","16 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on glimmerindex.com: 16 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crest-arcade-nonograms","name":"Crest Arcade Nonograms","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 91.7%, past 85% since November 2026."],["import.importer","pass","We read the board on glimmerindex.com: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crest-arcade-ravelin","name":"Crest Arcade Ravelin","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on glimmerindex.com: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crest-arcade-starjar","name":"Crest Arcade Starjar","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on glimmerindex.com: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crest-arcade-sudoku","name":"Crest Arcade Sudoku","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2026."],["import.importer","pass","We read the board on glimmerindex.com: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crest-arcade-wildhaven","name":"Crest Arcade Wildhaven","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since March 2025."],["import.importer","pass","We read the board on glimmerindex.com: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cresta-bench","name":"Cresta-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 82.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on fireworks.ai: 14 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crisismmd","name":"CrisisMMD","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on mimran.me: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"criticeval","name":"CriticEval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","12 models we track, plus 21 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-3.5 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on open-compass.github.io: 12 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"critpt","name":"CritPt","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries), launch posts (12 scores) and system cards (5 scores)."],["scores.models","pass","24 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 83.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 11 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"critpt-v2","name":"CritPt v2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 83.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 11 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crm-agent-bench","name":"CRM-Agent-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 83.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on crm-agent-bench.github.io: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crmarena","name":"CRMArena","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for DeepSeek-R1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 64.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 24 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crmarena-pro","name":"CRMArena-Pro","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#104)."],["scores.headroom","pass","The best is 56.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crochetbench","name":"CrochetBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","7 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash-Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 4 (#113)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 7 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cronos","name":"CRONOS","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for OpenVLA."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on embodiedai-ntu.github.io: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bccl-csa-2026","name":"Cross-lingual Sentiment Consistency Analysis for Classical Chinese Literature","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cross-lingual-test-dataset","name":"Cross-lingual Test Dataset","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cross-mr","name":"CroSS-MR","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","4 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Qwen1.5-14B-Chat."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 54.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 8 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crossmodal-3600","name":"Crossmodal-3600","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (52 entries)."],["scores.models","pass","1 model we track, plus 51 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MTEB: 1 row for models we track and 51 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crosspoint-bench","name":"CrossPoint-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries)."],["scores.models","pass","11 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Qwen3-VL-30B-A3B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 72.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 14 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crossre","name":"CrossRE","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crosssum","name":"CrossSum","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crossvid","name":"CrossVid","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","16 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4.1 (#138)."],["scores.headroom","pass","The best is 65%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 13 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crosswordbench-leng","name":"CrossWordBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for QwQ-32B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 70.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crows-pairs","name":"CrowS-Pairs","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Gemma 2 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 2 27B (#190)."],["scores.headroom","pass","The best is 51.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crpe","name":"CRPE","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries), launch posts (1 score) and system cards (4 scores)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Nemotron Nano 12B v2 VL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 79.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crsbench","name":"CRSBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and system cards (1 score)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from July 2026, for MAI-Cyber-1-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 82.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on oss-crs.openssf.org: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crust-bench","name":"CRUST-bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","7 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Claude Opus 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 48%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 22 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cruxeval","name":"CRUXEval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (35 entries), launch posts (6 scores) and system cards (4 scores)."],["scores.models","pass","20 models we track, plus 24 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Granite 4.1 30B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","info","Saturated. The best is 95.9%, past 85% since September 2025."],["import.importer","pass","We read the board on crux-eval.github.io: 15 rows for models we track and 24 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cryptanalysisbench","name":"CryptanalysisBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 34.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cryptanalysis-bench.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"crypto-ctf-anthropic","name":"Crypto CTF challenges (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Saturated. The best is 94.4%, past 85% since November 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cs-4k","name":"CS-4k","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (30 entries)."],["scores.models","pass","18 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 19 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cs-fleurs","name":"CS-FLEURS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"csat-math-2025","name":"CSAT Math 2025","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","8 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 20 months ago, in March 2025, for EXAONE Deep 32B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-R1 (#130)."],["scores.headroom","info","Saturated. The best is 94.5%, past 85% since January 2025."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"csatqa","name":"CSATQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"csl","name":"CSL","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"csr-bench","name":"CSR-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cstnews","name":"CSTNews","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cstutorbench","name":"CSTutorBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","8 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 31B IT."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 4 31B IT (#101)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 16 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"csvqa","name":"CSVQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries) and system cards (1 score)."],["scores.models","pass","13 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for STEP3-VL-10B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 63.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 12 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ct-dataset-1-google","name":"CT Dataset 1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ct-rate","name":"CT-RATE","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on nature.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ctf-archive","name":"CTF-Archive","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ctf-archive-diamond","name":"CTF-Archive Diamond","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from June 2026, for GLM-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 71%, under 85%, so it still separates models."],["import.importer","pass","We read the board on atlas.village.dev: 6 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cti-realm","name":"CTI-REALM","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 63.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ukgovernmentbeis.github.io: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cti-to-mitre","name":"CTI-to-MITRE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ctibench","name":"CTIBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","2 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Llama 3-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-70B (#182)."],["scores.headroom","pass","The best is 65.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on maveryn.github.io: 2 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ctkfacts","name":"CTKFacts","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (192 entries)."],["scores.models","pass","4 models we track, plus 188 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MTEB: 4 rows for models we track and 188 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-cua-bench","name":"CUA-bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 19.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 9 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cua-bench-eda-kicad","name":"Cua-Bench EDA (KiCad v1)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 24%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cua.ai: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cua-shade-arena","name":"CUA-SHADE-Arena","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cua-world-long","name":"CUA-World-Long","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 27.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cmu-l3.github.io: 7 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cuad","name":"CUAD","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on datasets-benchmarks-proceedings.neurips.cc: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cubebench","name":"CubeBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cuespeak","name":"CueSpeak","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for TML-Interaction-Small. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 81.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"culturalbench","name":"CulturalBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries) and system cards (1 score)."],["scores.models","pass","18 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Apertus 70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","pass","The best is 74.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 17 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"culturalriddles","name":"CulturalRiddles","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","20 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Apertus 1.5 70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 79.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 23 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"culturaqa","name":"CulturaQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"culturavivapt","name":"CulturaVivaPT","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","8 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Apertus 8B Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-8B (#154)."],["scores.headroom","pass","The best is 72%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 8 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cultureatlas","name":"CultureAtlas","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Llama 2-13B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-13B (#223)."],["scores.headroom","pass","The best is 73.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cure","name":"CURE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"curie-biogr","name":"CURIE Biodiversity Georeferencing","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","4 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Gemini 2.0 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on proceedings.iclr.cc: 4 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"curie-geo","name":"CURIE Geospatial Dataset Extraction","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","4 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Gemini 2.0 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"current-facts-writer","name":"Current-facts set","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cursorbench-2025","name":"Cursor Bench (2025 internal version)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Composer 1.5. That’s expected for a test replaced by CursorBench."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cursorbench","name":"CursorBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (3 entries) and system cards (1 score)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Composer 2. That’s expected for a test replaced by CursorBench 3.1."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 61.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cursor.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cursorbench-3-1","name":"CursorBench 3.1","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries), launch posts (3 scores) and system cards (2 scores)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 72.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on web.archive.org: 30 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 4 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cursorbench-3-2","name":"CursorBench 3.2","summary":{"pass":5,"warn":1,"fail":0,"info":4,"text":"5 of 6 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Fable 5.1."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","pass","The best is 73.4%, under 85%, so it still separates models."],["import.importer","warn","The record links a board on cursor.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cursorbench-4-0","name":"CursorBench 4.0","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries), launch posts (3 scores) and system cards (4 scores)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","5 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 65.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cursor.com: 53 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 4 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cursorbench-4-1","name":"CursorBench 4.1","summary":{"pass":11,"warn":1,"fail":0,"info":0,"text":"11 of 12 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (12 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","5 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 65.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cursor.com: 53 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cusp-scientific-forecasting","name":"CUSP","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","7 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 60.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"customer-support-qa-bytedance","name":"Customer Support Q&A (ByteDance Seed in-house)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Seed1.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 72.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cutcraft-benchmark","name":"CutCraft Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","5 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for LTX-2.3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cutverse","name":"CutVerse","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 68.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cv3-eval","name":"CV3-Eval","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen3.5-Omni-Plus."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cvbench","name":"CVBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries), launch posts (6 scores) and system cards (13 scores)."],["scores.models","pass","22 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Saturated. The best is 88%, past 85% since July 2025."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cvdp","name":"CVDP","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","25 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 64.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Mercor: 27 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cve-rediscovery-anthropic","name":"CVE Rediscovery (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 89.6%, past 85% since November 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cve-bench","name":"CVE-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries) and system cards (14 scores)."],["scores.models","pass","14 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Saturated. The best is 94.1%, past 85% since December 2025."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cve-bench-2","name":"CVE-Bench 2","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cvqa","name":"CVQA","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (26 entries) and system cards (1 score)."],["scores.models","pass","4 models we track, plus 23 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Aya Vision 32B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 75.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on eval.ai: 8 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cvtg-2k","name":"CVTG-2K","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for GLM-Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cwe-bench","name":"CWE-bench","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Carbon."],["scores.frontier","pass","5 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 71%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cwe-bench.com: 12 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cwe-bench-2","name":"CWE-bench 2","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","8 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 75%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cwe-bench.com: 20 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cweval","name":"CWEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries) and system cards (1 score)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3-Coder-Next."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (October 2024) (#150)."],["scores.headroom","pass","The best is 56.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cybench","name":"Cybench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries), launch posts (1 score), system cards (18 scores) and claims Epoch AI collected (9 models)."],["scores.models","pass","30 models we track."],["scores.recent","pass","The newest is from July 2026, for Muse Spark 1.1."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.7 (#31)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since February 2026."],["import.importer","pass","We read the board on cybench.github.io: 16 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 5 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyber-adaptive-jailbreak-eval-google","name":"Cyber Adaptive Jailbreak Eval","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyber-agentic-safeguard-eval-google","name":"Cyber agentic safeguard evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyber-conversational-coverage-eval-google","name":"Cyber Conversational Evals","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"epoch-eci-cyber","name":"Cyber ECI","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (75 entries)."],["scores.models","pass","65 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on epoch.ai: 65 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyber-harm-coverage-anthropic","name":"Cyber harm coverage (Anthropic)","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyber-key-skills-v1-google","name":"Cyber key skills benchmark v1","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Gemini 3 Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Saturated. The best is 91.7%, past 85% since November 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyber-key-skills-v2-google","name":"Cyber key skills benchmark v2","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Gemini 3 Pro. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyber-range-openai","name":"Cyber Range","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (10 scores)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from August 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Saturated. The best is 93.3%, past 85% since April 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyber-static-jailbreak-eval-google","name":"Cyber Static Jailbreak Eval","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyber-utility-openai","name":"Cyber Utility","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 78.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyber-harness-network-challenges-anthropic","name":"Cyber-harness network challenges (Incalmo cyber ranges)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 66.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyberbench-liu","name":"CyberBench (Liu et al.)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-cyberbench","name":"CyberBench (Vals)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 88.1%, past 85% since July 2026."],["import.importer","pass","We read the board on vals.ai: 26 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-cyberbench-1-1","name":"CyberBench v1.1 (Vals)","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (47 entries) and launch posts (1 score)."],["scores.models","pass","44 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Small."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 80.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 45 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cybergym","name":"CyberGym","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (103 entries), launch posts (17 scores) and system cards (23 scores)."],["scores.models","pass","37 models we track, plus 98 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Saturated. The best is 96%, past 85% since July 2026."],["import.importer","pass","We read the board on cybergym.io: 76 rows for models we track and 27 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cybergym-final-submission","name":"CyberGym (final-submission metric)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for MAI-Cyber-1-Flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 86.3%, past 85% since July 2026."],["import.importer","warn","The record links a board on cybergym.io that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cybergym-e2e","name":"CyberGym-E2E","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 89%, past 85% since July 2026."],["import.importer","pass","We read the board on cybergym.io: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cybergym-e2e-615","name":"CyberGym-E2E (615 tasks)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 22.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cybergym.io: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cybermetric","name":"CyberMetric","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (25 entries)."],["scores.models","pass","11 models we track, plus 14 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for phi-3-mini 3.8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is phi-3-mini 3.8B (#204)."],["scores.headroom","pass","The best is 74.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 11 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-cyberoffense","name":"CyberOffense","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 87.7%, past 85% since September 2026."],["import.importer","pass","We read the board on vals.ai: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyberseceval-autocomplete","name":"CyberSecEval Autocomplete","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries) and system cards (3 scores)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Muse Spark 1.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.1 (#48)."],["scores.headroom","pass","The best is 73.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyberseceval-autonomous-offensive-cyber-operations","name":"CyberSecEval Autonomous Offensive Cyber Operations","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyberseceval-interpreter-abuse","name":"CyberSecEval Code Interpreter Abuse","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries) and system cards (2 scores)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Muse Spark 1.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyberseceval-instruct","name":"CyberSecEval Instruct","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries) and system cards (3 scores)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Muse Spark 1.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.1 (#48)."],["scores.headroom","pass","The best is 77.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyberseceval-mitre","name":"CyberSecEval MITRE","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","2 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in August 2023, for Code Llama 34B Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-13B (#223)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on huggingface.co: 2 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyberseceval-mitre-frr","name":"CyberSecEval MITRE False Refusal Rate","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","1 model we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in August 2023, for Code Llama 34B Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 9 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyberseceval-prompt-injection","name":"CyberSecEval Prompt Injection","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyberseceval-spear-phishing","name":"CyberSecEval Spear Phishing","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyberseceval-visual-prompt-injection","name":"CyberSecEval Visual Prompt Injection","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyberseceval-vulnerability-exploitation","name":"CyberSecEval Vulnerability Exploitation","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cybersecurity-privacy-legal-agent-eval","name":"Cybersecurity and Privacy Legal Agent Eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cybersoceval-malware-analysis","name":"CyberSOCEval Malware Analysis","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries) and system cards (1 score)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from July 2026, for MAI-Cyber-1-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 33.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cybersoceval-threat-intel-reasoning","name":"CyberSOCEval Threat Intelligence Reasoning","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for MAI-Cyber-1-Flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 55.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyberstrike-turing","name":"CyberStrike","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 31.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.turing.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyc3d","name":"Cyc3D","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cypherbench","name":"CypherBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Llama 3.2 3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","pass","The best is 61.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyrillic-turkic-language-identification","name":"Cyrillic Turkic language identification","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyscenariobench","name":"CyScenarioBench","summary":{"pass":6,"warn":1,"fail":0,"info":3,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (19 scores)."],["scores.models","pass","18 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since October 2026."],["import.importer","warn","The record links a board on irregular.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"czechbankqa","name":"CzechBankQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"d2k-bench","name":"D2K-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dabstep","name":"DABstep","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","28 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 19 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dacomp","name":"DAComp","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for DeepSeek-V3.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 47%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dacsa","name":"DACSA","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"daily-omni","name":"Daily-Omni","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries), launch posts (7 scores) and system cards (5 scores)."],["scores.models","pass","17 models we track, plus 16 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","info","Saturated. The best is 85.1%, past 85% since September 2026."],["import.importer","pass","We read the board on github.com: 22 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 4 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dailyreport-meituan","name":"DailyReport","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","7 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dala","name":"DaLA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (108 entries)."],["scores.models","pass","25 models we track, plus 83 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-9B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5-9B (#128)."],["scores.headroom","pass","The best is 61.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on euroeval.com: 29 rows for models we track and 83 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dalaj","name":"DaLAJ","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on datawrapper.dwcdn.net: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dalaj-ged-2","name":"DaLAJ-GED 2.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on lab.kb.se: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"daleel-2026","name":"Daleel 2026","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dane","name":"DaNE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"danfever","name":"DanFEVER","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"art-dangerous-content-violations-google","name":"Dangerous Content policy violations (from ART)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dangerous-content-safety-eval-google","name":"Dangerous Content Safety","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 96.6%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"danish-semantic-reasoning-benchmark","name":"Danish Semantic Reasoning Benchmark","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dansk","name":"DANSK","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"danske-talemader","name":"Danske Talemåder","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dans-editorial-checks-every","name":"Dan’s Editorial Checks","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 86%, past 85% since September 2026."],["import.importer","pass","We read the board on checks.every.to: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dare-bench","name":"DARE-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 49.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"darijabench","name":"DarijaBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"darijahellaswag","name":"DarijaHellaSwag","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"darijammlu","name":"DarijaMMLU","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dashbench","name":"DashBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","11 models we track, plus 14 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 34.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cornercase-dashbench.github.io: 11 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"data-agent-benchmark","name":"Data Agent Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (54 entries)."],["scores.models","pass","5 models we track, plus 49 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen3.8-Flash-Next."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 70.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 41 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"data-analysis-benchmark-stepfun","name":"Data Analysis Benchmark (StepFun)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Step 3.5 Flash. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 39.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legalbenchmarks-data-extraction-applications","name":"Data Extraction (Applications)","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on legalbenchmarks.ai: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legalbenchmarks-data-extraction-models","name":"Data Extraction (Models)","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 76.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on legalbenchmarks.ai: 27 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"data-science-tasks-openai","name":"Data Science Tasks (OpenAI)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 40.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snowflake-data-eng-bench","name":"data-eng-bench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on hub.harborframework.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dataanalyticsbench-plotly","name":"DataAnalyticsBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (54 entries)."],["scores.models","pass","51 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on plotly.com: 51 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"database-migration-tasks-openai","name":"Database Migration Tasks (OpenAI)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 63.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"databench-cardiff","name":"DataBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","2 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Llama 2-13B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-13B (#223)."],["scores.headroom","pass","The best is 20.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hex-databench-v1-1","name":"DataBench v1.1","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","21 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 70.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on hex.tech: 83 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"databricks-ai-index","name":"Databricks AI Index","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","19 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on developers.databricks.com: 19 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"datacenterbench-tier-1","name":"DataCenterBench Tier 1","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 40%, under 85%, so it still separates models."],["import.importer","pass","We read the board on robocurve.org: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dataclue","name":"DataCLUE","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on github.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"datacomp-image-encoder-evaluation","name":"DataComp image encoder evaluation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on datacomp.ai: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dataprep-bench","name":"DataPrep-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on datapreparationbench.github.io: 20 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dataprep-bench-finance","name":"DataPrep-Bench Finance","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on datapreparationbench.github.io: 16 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dataprep-bench-general","name":"DataPrep-Bench General","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on datapreparationbench.github.io: 16 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dataprep-bench-law","name":"DataPrep-Bench Law","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on datapreparationbench.github.io: 16 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dataprep-bench-math","name":"DataPrep-Bench Math","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on datapreparationbench.github.io: 16 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dataprep-bench-medical","name":"DataPrep-Bench Medical","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on datapreparationbench.github.io: 16 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dataprep-bench-science","name":"DataPrep-Bench Science","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on datapreparationbench.github.io: 16 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"datascibench","name":"DataSciBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"datbench-vlm","name":"DatBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (26 entries)."],["scores.models","pass","22 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Qwen3-VL-2B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 64%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 22 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"davis-2017","name":"DAVIS 2017","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on davischallenge.org: 20 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"daxbench-2","name":"DAXBench v2","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (50 entries)."],["scores.models","pass","49 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 95.9%, past 85% since April 2026."],["import.importer","pass","We read the board on daxbench.com: 49 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"daybreak-blue-cyber-defense-openai","name":"Daybreak Blue and Enabling Cyber Defense","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dayjob-accounting","name":"DAYJOB: Accounting","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 49.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on fireworks.ai: 26 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dayjob-data-science","name":"DAYJOB: Data Science","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 37.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on fireworks.ai: 16 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dayjob-finance","name":"DAYJOB: Finance","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries) and launch posts (1 score)."],["scores.models","pass","40 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Small."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 42.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on surgehq.ai: 40 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dayjob-healthcare","name":"DAYJOB: Healthcare","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (38 entries)."],["scores.models","pass","38 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 48.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on surgehq.ai: 38 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dayjob-law","name":"DAYJOB: Law","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","22 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 41.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on fireworks.ai: 22 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dayjob-tax","name":"DAYJOB: Tax","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 10.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on surgehq.ai: 16 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dbpedia-14","name":"DBpedia 14","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (195 entries)."],["scores.models","pass","5 models we track, plus 190 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 99.3%, past 85% since June 2025."],["import.importer","pass","We read the board on MTEB: 5 rows for models we track and 190 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dcase-foley-sound-synthesis-2023","name":"DCASE 2023 Foley Sound Synthesis","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on dcase.community: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dcase-2024-language-queried-audio-source-separation","name":"DCASE 2024 Language-Queried Audio Source Separation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on dcase.community: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dcase-sound-scene-synthesis-2024","name":"DCASE 2024 Sound Scene Synthesis","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on dcase.community: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dcase-2025-audio-question-answering","name":"DCASE 2025 Audio Question Answering","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","3 models we track, plus 28 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Kimi-Audio-7B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","pass","The best is 52.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on dcase.community: 3 rows for models we track and 28 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dcs-2018-sanskrit-word-segmentation","name":"DCS 2018 Sanskrit Word Segmentation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ddiextraction-2013","name":"DDIExtraction 2013","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ddisco","name":"DDisCo","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ddxplus","name":"DDXPlus","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on proceedings.neurips.cc: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"de-novo-protein-binder-design-anthropic","name":"De novo protein binder design (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 85.1%, past 85% since November 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dealbench-100","name":"DealBench-100","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"debagreement","name":"DEBAGREEMENT","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","An importer exists, but it hasn’t read the board yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"debugbench","name":"DebugBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 3 years ago, in June 2023, for GPT-3.5 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","pass","The best is 75%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"decbench","name":"DecBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 51.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on decbench.com: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"decepeval","name":"DecepEval","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"representative-prompts-deception-openai","name":"Deception Evaluations with Representative Prompts","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"production-deception-openai","name":"Deception in production traffic","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 11 months ago, in December 2025, for GPT-5.2. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deceptionbench","name":"DeceptionBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries) and system cards (2 scores)."],["scores.models","pass","12 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Muse Spark 1.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.1 (#48)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 50 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deceptionbench-pku-alignment","name":"DeceptionBench (PKU Alignment)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for DeepSeek-R1-Distill-Qwen-7B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"d-rex-gray-swan","name":"Deceptive Reasoning Exposure Suite","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"decide","name":"DECIDE","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","9 models we track, plus 11 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for MiMo-V2.6 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark (#54)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on benchmark.tazolabs.com: 9 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"decision-index","name":"Decision Index","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (323 entries)."],["scores.models","pass","3 models we track, plus 320 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 27B (#59)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track and 320 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"decision-index-v0-3","name":"Decision Index Public v0.3","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Granite Switch 4.1 30B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"decision-index-vision","name":"Decision Index Vision","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (95 entries)."],["scores.models","pass","2 models we track, plus 93 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5 397B-A17B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5 397B-A17B (#78)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 2 rows for models we track and 93 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"decision-model-benchmark-arize","name":"Decision Model Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","1 model we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Luna (#38)."],["scores.headroom","info","Saturated. The best is 87.9%, past 85% since September 2026."],["import.importer","pass","We read the board on arize.com: 1 row for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deck-bench-moonshot","name":"DECK Bench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 73.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deck-arena-moonshot","name":"DECK-Arena","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3.1 Pro (#17)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"exploitgym-honeypot-openai","name":"Declining to Exploit a Honeypot During Difficult ExploitGym Problems","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (9 scores)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fab-decodability-assessment","name":"Decodability Assessment Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","30 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 95.9%, past 85% since December 2025."],["import.importer","pass","We read the board on fab-ai.org: 30 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"decodingtrust","name":"DecodingTrust","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (55 entries)."],["scores.models","pass","5 models we track, plus 50 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in December 2023, for Gemini 1.0 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on decodingtrust.github.io: 5 rows for models we track and 50 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deep-memory-retrieval","name":"Deep Memory Retrieval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deep-research-bench-futuresearch","name":"Deep Research Bench (FutureSearch)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","21 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on evals.futuresearch.ai: 36 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-deep-research-bench-moonshot","name":"Deep Research Bench (Moonshot)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Saturated. The best is 90%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deep-research-prompt-injection-openai","name":"Deep research prompt injection evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 21 months ago, in February 2025, for OpenAI deep research. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deep20bench-1-1","name":"Deep20Bench 1.1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","25 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Sol."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on deep20bench.com: 25 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepambigqa","name":"DeepAmbigQA","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepconsult","name":"DeepConsult","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Seed1.8."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 61.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepfinance","name":"DeepFinance","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","28 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 62%, under 85%, so it still separates models."],["import.importer","pass","We read the board on juliusresearch.ai: 28 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepgram-voice-ai-benchmarks","name":"Deepgram Voice AI Benchmarks","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on deepgram.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepmind-control-vision-benchmark","name":"DeepMind Control Vision Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepmind-dangerous-capability-evaluations","name":"DeepMind dangerous capability evaluations","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in December 2023, for Gemini 1.0 Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 20.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepmind-fast-language-learning-tasks","name":"DeepMind Fast Language Learning Tasks","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepmind-mathematics","name":"DeepMind Mathematics","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores) and system cards (1 score)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Granite 4.1 30B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 81.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepphy","name":"DeepPHY","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","15 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 39%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 30 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepplanning","name":"DeepPlanning","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries) and launch posts (10 scores)."],["scores.models","pass","25 models we track."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 62.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on qwenlm.github.io: 22 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepresearch-arena","name":"DeepResearch Arena","summary":{"pass":6,"warn":3,"fail":0,"info":3,"text":"6 of 9 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Grok 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4 (#86)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepresearch-bench","name":"DeepResearch Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries), boards Epoch AI copied (22 models), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","25 models we track, plus 18 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 55.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 18 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepresearch-bench-ii","name":"DeepResearch Bench II","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","1 model we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for OpenAI deep research."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 43.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 12 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepresearch-bench-ii-gemini25","name":"DeepResearch Bench II (Gemini 2.5 Pro judge)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","1 model we track, plus 18 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for OpenAI deep research."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 45.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 18 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepresearcheval","name":"DeepResearchEval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on infinity-ailab.github.io: 7 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepresearcheval-1","name":"DeepResearchEval v1","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on infinity-ailab.github.io: 9 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deeprsi","name":"DeepRSI","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on deeprsi.com: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepsdo-description","name":"DeepSDO Description","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepsearchqa","name":"DeepSearchQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries), launch posts (16 scores) and system cards (13 scores)."],["scores.models","pass","35 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 95%, past 85% since February 2026."],["import.importer","pass","We read the board on kaggle.com: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepsecbench","name":"DeepsecBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","31 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on vercel.com: 60 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepseek-chinese-creative-writing-eval","name":"DeepSeek Chinese creative writing evaluation","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepseek-chinese-functional-writing-eval","name":"DeepSeek Chinese functional writing evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is DeepSeek-V4-Pro (#70)."],["scores.headroom","pass","The best is 62.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepseek-white-collar-tasks-eval","name":"DeepSeek Chinese white-collar tasks evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is DeepSeek-V4-Pro (#70)."],["scores.headroom","pass","The best is 53%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepseek-complex-if-multiturn-writing-eval","name":"DeepSeek complex instruction following and multi-turn writing evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is DeepSeek-V4-Pro (#70)."],["scores.headroom","pass","The best is 45.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepseek-in-house-safety-eval","name":"DeepSeek in-house safety benchmark","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for DeepSeek-R1. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is DeepSeek-R1 (#130)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepseek-jailbreak-eval","name":"DeepSeek jailbreak robustness evaluation","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for DeepSeek-R1. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is DeepSeek-R1 (#130)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepseek-llm-in-house-safety-eval","name":"DeepSeek LLM in-house safety evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in January 2024, for DeepSeek LLM 67B. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepseek-multilingual-safety-eval","name":"DeepSeek multilingual safety evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for DeepSeek-R1. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is DeepSeek-R1 (#130)."],["scores.headroom","pass","The best is 74.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepseek-rnd-coding-benchmark","name":"DeepSeek R&D Coding Benchmark","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is DeepSeek-V4-Pro (#70)."],["scores.headroom","pass","The best is 67%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepseek-search-qa-eval","name":"DeepSeek search Q&A evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is DeepSeek-V4-Pro (#70)."],["scores.headroom","pass","The best is 28.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepshop","name":"DeepShop","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","3 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for MolmoWeb 8B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 42.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepswe","name":"DeepSWE","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries), launch posts (54 scores) and system cards (21 scores)."],["scores.models","pass","70 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 79%, under 85%, so it still separates models."],["import.importer","pass","We read the board on deepswe.datacurve.ai: 70 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 17 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepsynth","name":"DEEPSYNTH","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","6 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","pass","The best is 8.97%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 9 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepterminalbench","name":"DeepTerminalBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 34%, under 85%, so it still separates models."],["import.importer","pass","We read the board on unipat.ai: 9 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"default-safety-cohere","name":"Default safety evaluation (Cohere)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Cohere Command A. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 70.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"defects4j","name":"Defects4J","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deference-to-governments-in-local-languages-anthropic","name":"Deference to governments in local languages (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"degennaro-workplace-sample","name":"DeGennaro Workplace sample","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"delegate-52","name":"DELEGATE-52","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"delta-dutch-legal-research","name":"DELTA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","23 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on legalbenchmarks.ai: 23 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"delulu","name":"Delulu","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","5 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Qwen2.5-Coder-32B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is StarCoder 2 15B (#224)."],["scores.headroom","pass","The best is 84.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"delusional-reinforcement-eval-google","name":"Delusional Reinforcement evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 94%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smm4h-2025-dementia-family-caregivers","name":"Dementia Family Caregiver Detection","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"demosthenes-argument-classification","name":"Demosthenes Argument Classification","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dense-200-bbox","name":"Dense 200 (bbox)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries) and launch posts (2 scores)."],["scores.models","pass","9 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Mistral Large 4."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","info","Saturated. The best is 87%, past 85% since August 2026."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dependency-bench","name":"dependency-bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Haiku 5.5 (#20)."],["scores.headroom","info","Saturated. The best is 85%, past 85% since October 2026."],["import.importer","pass","We read the board on kaggle.com: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deployguard","name":"DeployGuard","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.5 Flash-Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Mini (#74)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deputybench","name":"DeputyBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Doubao Seed Evolving."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 92.6%, past 85% since September 2026."],["import.importer","pass","We read the board on arxiv.org: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"der2-bench","name":"DeR2 Bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Seed2.0 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 83.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on retrieval-infused-reasoning-sandbox.github.io: 20 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"describable-textures-dataset","name":"Describable Textures Dataset","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on openaccess.thecvf.com: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"desed","name":"DESED","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on project.inria.fr: 11 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"design-bench-moonshot","name":"Design Bench (Moonshot)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3.1 Pro (#17)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"design-tasks-openai","name":"Design Tasks (OpenAI)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 50%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"design2code","name":"Design2Code","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries), launch posts (5 scores) and system cards (5 scores)."],["scores.models","pass","7 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GLM-5V-Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 15 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-3d","name":"DesignArena 3D Design","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (159 entries) and launch posts (1 score)."],["scores.models","pass","144 models we track, plus 15 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 167 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-agentic-game-dev","name":"DesignArena Agentic Game Dev","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","29 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 30 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-agentic-image-to-html","name":"DesignArena Agentic Image to HTML","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Grok 4.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 11 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-agentic-overall","name":"DesignArena Agentic Overall","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (80 entries)."],["scores.models","pass","80 models we track."],["scores.recent","pass","The newest is from November 2026, for Qwen4-Max."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 86 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-android","name":"DesignArena Android (Kotlin)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (46 entries)."],["scores.models","pass","46 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 48 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-ascii","name":"DesignArena ASCII Art","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (86 entries)."],["scores.models","pass","82 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 98 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-audio-realism-bench","name":"DesignArena Audio Realism Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","11 models we track, plus 23 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Voxtral Mini TTS 3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 11 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-blender","name":"DesignArena Blender","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","14 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 18 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-builder","name":"DesignArena Builder","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 18 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena","name":"DesignArena Code","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (174 entries)."],["scores.models","pass","153 models we track, plus 21 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 177 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-code-to-video","name":"DesignArena Code to Video","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","33 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 34 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-data-viz","name":"DesignArena Data Visualization","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (135 entries)."],["scores.models","pass","126 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Muse Ember."],["scores.frontier","pass","8 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 148 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-web-apps","name":"DesignArena Frontend Web App","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (53 entries) and launch posts (1 score)."],["scores.models","pass","53 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Small."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 56 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-fullstack","name":"DesignArena Full Stack","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (50 entries)."],["scores.models","pass","50 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 52 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-game-dev","name":"DesignArena Game Dev","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (175 entries) and launch posts (1 score)."],["scores.models","pass","154 models we track, plus 21 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 177 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-godot-game-dev","name":"DesignArena Godot Game Dev","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 27 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-graphic-design","name":"DesignArena Graphic Design","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (76 entries)."],["scores.models","pass","46 models we track, plus 30 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Flash Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 47 rows for models we track and 30 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-html-slides","name":"DesignArena HTML Slides","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","28 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 29 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-image","name":"DesignArena Image","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (88 entries)."],["scores.models","pass","52 models we track, plus 36 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-Image-2.6 Flare."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 56 rows for models we track and 36 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-image-editing","name":"DesignArena Image Editing","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (56 entries)."],["scores.models","pass","23 models we track, plus 33 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-Image-2.6 Flare."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 28 rows for models we track and 33 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-image-to-frontend","name":"DesignArena Image to Frontend","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","33 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 34 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-image-to-html","name":"DesignArena Image to HTML","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (55 entries)."],["scores.models","pass","53 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 56 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-image-to-video","name":"DesignArena Image to Video","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","14 models we track, plus 28 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini Omni 1.2 Flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 15 rows for models we track and 28 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-logo","name":"DesignArena Logo","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (77 entries)."],["scores.models","pass","47 models we track, plus 30 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Flash Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 48 rows for models we track and 30 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-mobile-builder","name":"DesignArena Mobile Builder","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 5 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-multi-image-to-video","name":"DesignArena Multi-Image to Video","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","8 models we track, plus 17 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini Omni 1.2 Flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 9 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-multimodal-to-video","name":"DesignArena Multimodal to Video","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","6 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini Omni 1.2 Flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 7 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-pdf","name":"DesignArena PDF","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (48 entries)."],["scores.models","pass","48 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 50 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-slides","name":"DesignArena Proprietary Harness Slides","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 13 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-python-pptx-slides","name":"DesignArena Python-PPTX Slides","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries)."],["scores.models","pass","37 models we track."],["scores.recent","pass","The newest is from November 2026, for Qwen4-Max."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 39 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-react-native","name":"DesignArena React Native","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (58 entries)."],["scores.models","pass","58 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 60 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-speech-to-speech","name":"DesignArena Speech-to-Speech","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","warn","The record links a board on designarena.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-spreadsheet","name":"DesignArena Spreadsheet","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries)."],["scores.models","pass","40 models we track."],["scores.recent","pass","The newest is from November 2026, for Qwen4-Max."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 42 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-svg","name":"DesignArena SVG","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (122 entries)."],["scores.models","pass","110 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","8 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 130 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-tts","name":"DesignArena Text-to-Speech","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries)."],["scores.models","pass","11 models we track, plus 25 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Voxtral Mini TTS 3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 11 rows for models we track and 25 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-ui-component","name":"DesignArena UI Component","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (166 entries)."],["scores.models","pass","149 models we track, plus 17 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 172 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-video","name":"DesignArena Video Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (48 entries)."],["scores.models","pass","18 models we track, plus 30 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini Omni 1.2 Flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 19 rows for models we track and 30 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-video-editing","name":"DesignArena Video Editing","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","5 models we track, plus 15 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 6 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-video-to-website","name":"DesignArena Video to Website","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Wonder (#16)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 20 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designarena-website","name":"DesignArena Website","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (184 entries) and launch posts (2 scores)."],["scores.models","pass","162 models we track, plus 22 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on designarena.ai: 188 rows for models we track and 22 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designjanus","name":"DesignJanus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"designqa","name":"DesignQA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on design-qa.github.io: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deskcraft","name":"DeskCraft","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries)."],["scores.models","pass","12 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Kimi K2.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 29.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"destructive-action-avoidance-openai","name":"Destructive action avoidance","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (11 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Saturated. The best is 90%, past 85% since February 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"destructive-action-reversion-openai","name":"Destructive action reversion on long rollouts","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 7 months ago, in April 2026, for GPT-5.5. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"destructive-actions-anthropic","name":"Destructive actions (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"destructive-actions-thinking-machines","name":"Destructive actions (Thinking Machines)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"destructive-production-evaluation-anthropic","name":"Destructive production evaluation (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Opus 4.7. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.7 (#31)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deta-studio-construction-takeoff","name":"Deta Studio AI Construction Takeoff","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on deta.studio: 67 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deal-astrophysics-ner","name":"Detecting Entities in the Astrophysics Literature","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codalab.lisn.upsaclay.fr: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deterministic-agent-reasoning-formatting-suite","name":"Deterministic Agent Reasoning & Formatting Suite","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2026."],["import.importer","pass","We read the board on kaggle.com: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"detests-dis-2024","name":"DETESTS-Dis 2024","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on detests-dis.github.io: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"detour","name":"DETOUR","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"devbench","name":"DevBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","3 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-4 Turbo (Jan 2024)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","pass","The best is 41.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"devbench-acceptance-testing","name":"DevBench Acceptance Testing","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-3.5 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","pass","The best is 29.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"devbench-environment-setup","name":"DevBench Environment Setup","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-3.5 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","pass","The best is 41.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"devbench-implementation","name":"DevBench Implementation","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","3 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-4 Turbo (Jan 2024)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","pass","The best is 7.10%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"devbench-unit-testing","name":"DevBench Unit Testing","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-3.5 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","pass","The best is 36.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"firecrawl-devdex","name":"Developer Retrieval Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on firecrawl.dev: 8 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"devops-gym","name":"DevOps-Gym","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","1 model we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Claude Sonnet 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 4 (#113)."],["scores.headroom","pass","The best is 27.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on devops-gym.com: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"devopsbench-100","name":"DevOpsBench-100","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on blobfish.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dew-logiq","name":"DEW-LogiQ","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","info","Saturated. The best is 87.4%, past 85% since April 2025."],["import.importer","pass","We read the board on dsiweb.cse.msu.edu: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dexarena","name":"DexArena","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for π0.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 52.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sipair-elastic.github.io: 3 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dexverse","name":"DexVerse","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for π0.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 34%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ycyao216.github.io: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deysi-spam-detection","name":"Deysi Spam Detection","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dfbench","name":"dfbench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","18 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 69%, under 85%, so it still separates models."],["import.importer","pass","We read the board on depthfirst.com: 32 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dfki-english-russian-wmt25","name":"DFKI English-Russian Linguistic Test Suite at WMT25","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","1 model we track, plus 9 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for GPT-4.1. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4.1 (#138)."],["scores.headroom","info","Saturated. The best is 89.5%, past 85% since April 2025."],["import.importer","pass","We read the board on aclanthology.org: 1 row for models we track and 9 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dgcl-reward-hacking","name":"DGCL Reward Hacking","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"di-bench","name":"DI-Bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 57.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"diabla","name":"DiaBLa","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (194 entries)."],["scores.models","pass","5 models we track, plus 189 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MTEB: 5 rows for models we track and 189 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"diagtrace","name":"DiagTrace","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Kimi K2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K2.5 (#76)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dialectsenteval-2026","name":"DialectSentEval 2026","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dictatorship-eval","name":"Dictatorship Eval","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 91.3%, past 85% since April 2026."],["import.importer","pass","We read the board on dictatoreval.org: 20 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"didemo","name":"DiDeMo","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dig-bench","name":"DiG-bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 71.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on digbench.ai: 8 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"digiklausur-short-answer-grading","name":"DigiKlausur Assisted Short Answer Grading","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkel-digital-work-index","name":"Digital Work Index","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on snorkel.ai: 16 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dihard-iii","name":"DIHARD III","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on dihardchallenge.github.io: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"diligencebench","name":"DiligenceBench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 58.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dillen-herbarium-labels","name":"Dillen herbarium specimen label benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"diode","name":"DIODE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on sota2.com: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dior","name":"DIOR","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 12 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dior-rsvg","name":"DIOR-RSVG","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dirty-integration","name":"Dirty Integration","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"representative-prompts-disallowed-content-openai","name":"Disallowed Content Evaluations with Representative Prompts","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"disbench","name":"DISBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 28.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"discfinllm-reranking","name":"DISCFinLLMReranking","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"discharge-me","name":"Discharge Me!","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on docs.google.com: 17 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"discomat","name":"DiSCoMaT","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"discourseee","name":"DiscourseEE","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Llama 3-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-70B (#182)."],["scores.headroom","pass","The best is 36.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 8 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"discoverable-memorization-google","name":"Discoverable memorization of training data","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"discoverybench-ai2","name":"DiscoveryBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"discox","name":"DiscoX","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","7 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Seed1.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on randomtutu.github.io: 7 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"discrim-eval","name":"Discrim-Eval","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for o3-mini."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"disengagement-aware-student-simulators","name":"Disengagement-Aware Student Simulators","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"disordered-eating-evaluations-anthropic","name":"Disordered eating evaluations (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voice-input-disparate-performance-openai","name":"Disparate performance on voice inputs","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"displace-2023","name":"DISPLACE Challenge 2023","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","warn","The record links a board on codalab.lisn.upsaclay.fr that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"displace-2024","name":"DISPLACE Challenge 2024","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on isca-archive.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"displace-m-2026","name":"DISPLACE-M 2026","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"distemist","name":"DisTEMIST","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on participants-area.bioasq.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"distressbench","name":"DistressBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (43 entries) and launch posts (1 score)."],["scores.models","pass","43 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Small."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 88.4%, past 85% since August 2026."],["import.importer","pass","We read the board on labs.scale.com: 43 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"divergence-attack-memorization-google","name":"Divergence attack (extractable memorization)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"diversevul","name":"DiverseVul","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Gemma 2 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 2 27B (#190)."],["scores.headroom","pass","The best is 57%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"diving48","name":"Diving48","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on openaccess.thecvf.com: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dna-design-tf-binding-openai","name":"DNA sequence design for transcription factor binding","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 16.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dna-synthesis-screening-evasion-anthropic","name":"DNA Synthesis Screening Evasion","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"do-you-see-me","name":"Do You See Me","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries) and system cards (1 score)."],["scores.models","pass","7 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for STEP3-VL-10B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 67.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 6 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"do-not-answer","name":"Do-Not-Answer","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and system cards (2 scores)."],["scores.models","pass","2 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for DeepSeek-R1. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-R1 (#130)."],["scores.headroom","info","Saturated. The best is 97.8%, past 85% since January 2024."],["import.importer","pass","We read the board on github.com: 6 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"do-anything-now","name":"DoAnythingNow (DAN)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Olmo 3.1 32B Think."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 54.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"doc2harness","name":"Doc2Harness","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 53%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 19 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"docci","name":"Docci","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini Embedding 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"docai-fabric-document-ai-benchmark-arena","name":"Document AI Benchmark Arena","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Saturated. The best is 92.6%, past 85% since January 2025."],["import.importer","pass","We read the board on docaifabric.com: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"document-arena","name":"Document Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries)."],["scores.models","pass","40 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arena.ai: 46 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"docvqa","name":"DocVQA","summary":{"pass":7,"warn":1,"fail":1,"info":3,"text":"7 of 9 checks pass, 1 warning, 1 problem"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries), launch posts (29 scores) and system cards (23 scores)."],["scores.models","pass","36 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Saturated. The best is 96.9%, past 85% since December 2023."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","warn","November 11th, 2026, 10 days before the data’s date."],["import.failed","fail","The read on November 18th, 2026 failed with “Command failed: curl --fail --silent --show-error --max-time 40 --user-agent benchmark-almanac (+https://gith…”. Pages show the rows from November 11th, 2026."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"doe-sample-pack","name":"DOE sample pack","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"does-the-answer-survive-disagreement","name":"Does the answer survive disagreement?","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for GLM-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GLM-5 (#89)."],["scores.headroom","pass","The best is 0%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dojo-mini-bench","name":"Dojo Mini Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dollar-street","name":"Dollar Street","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mdolly-v2","name":"Dolly Machine Translated v2","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dolphinbench","name":"DolphinBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 5 (#35)."],["scores.headroom","pass","The best is 70.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on dolphinbench.ai: 13 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"domain-dynamics-english-hindi","name":"Domain Dynamics English-Hindi Translation","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","3 models we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for Command R+."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-70B (#182)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dibs-databricks","name":"Domain Intelligence Benchmark Suite","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dosimeter-ai-disinformation-benchmark","name":"Dosimeter AI Disinformation Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on dosimeteraudit.org: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dosimeter-us-midterm-source-audit-2026","name":"Dosimeter US midterm source audit","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on dosimeteraudit.org: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dotebench","name":"doteBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"doubao-multi-turn-bench-bytedance","name":"Doubao Multi-Turn Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 62.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on seed.bytedance.com: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dp-bench","name":"DP-Bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 7 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dpai-arena-spring-benchmark","name":"DPAI Arena Spring Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","7 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on dpaia.dev: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dpg-bench","name":"DPG-Bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","2 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for GLM-Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 84.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"draco","name":"DRACO","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries), launch posts (4 scores) and system cards (8 scores)."],["scores.models","pass","14 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 88.6%, past 85% since July 2026."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drafterbench","name":"DrafterBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","17 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 84.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 17 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"draftnepabench","name":"DraftNEPABench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drama-eval","name":"DRAMA-Eval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","2 models we track, plus 15 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on aslp-lab.github.io: 2 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dravidianlangtech-tulu-hope-speech-2026","name":"DravidianLangTech 2026 Hope Speech Detection in Code-Mixed Tulu","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dravidianlangtech-2026-multilevel-political-memes","name":"DravidianLangTech 2026 Multilevel Political Meme Classification in Tamil and Malayalam","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dravidianlangtech-tamil-dialect-speech-2026","name":"DravidianLangTech 2026 Tamil Dialect Speech Recognition and Classification","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drawbench","name":"DrawBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drawing-with-llms-kaggle","name":"Drawing with LLMs","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on kaggle.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drawingvqa","name":"DrawingVQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","18 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 77.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on joonv2.github.io: 18 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drbench","name":"DRBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","5 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for DeepSeek-V3.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 15 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dreaddit","name":"Dreaddit","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dream-1k","name":"DREAM-1K","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","9 models we track, plus 23 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Seed1.5-VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o mini (#178)."],["scores.headroom","pass","The best is 43.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tarsier-vlm.github.io: 8 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dreamgen-bench","name":"DreamGen Bench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dress-essay-scoring","name":"DREsS","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-8B (#203)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drivebench","name":"DriveBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","7 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for Qwen2-VL-72B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 21 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drivebench-behavior","name":"DriveBench Behavior","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","3 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for Qwen2-VL-7B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drivebench-perception","name":"DriveBench Perception","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","3 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for Qwen2-VL-7B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drivebench-planning","name":"DriveBench Planning","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","3 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for Qwen2-VL-7B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drivebench-prediction","name":"DriveBench Prediction","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","3 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for Qwen2-VL-7B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drivingbench","name":"DrivingBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2026."],["import.importer","pass","We read the board on drivingbench.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drivingbench-v2","name":"DrivingBench v2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","6 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 89.5%, past 85% since October 2026."],["import.importer","pass","We read the board on drivingbench.com: 8 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"droidtask","name":"DroidTask","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on autodroid-sys.github.io: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drone-bench","name":"Drone-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 97.3%, past 85% since July 2026."],["import.importer","pass","We read the board on andonlabs.com: 19 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drop","name":"DROP","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries), launch posts (35 scores) and system cards (39 scores)."],["scores.models","pass","41 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Saturated. The best is 93.5%, past 85% since May 2023."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drugdiscoverybench","name":"DrugDiscoveryBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 71.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 33 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drugprot","name":"DrugProt","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"drugtargetworld","name":"DrugTargetWorld","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on drugtargetworld.vercel.app: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ds-1000","name":"DS-1000","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for GPT-4 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#172)."],["scores.headroom","pass","The best is 53.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ds-arena-code","name":"DS-Arena-Code","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries) and launch posts (1 score)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for DeepSeek-V2.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 63.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on api-docs.deepseek.com: 1 row for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ds-fim-eval","name":"DS-FIM-Eval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries) and launch posts (1 score)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for DeepSeek-V2.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 78.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cdn.deepseek.com: 1 row for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dsbench","name":"DSBench","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for ChatGPT agent. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 89.9%, past 85% since July 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dsbench-deepseek-internal","name":"DSBench (DeepSeek internal)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (5 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from August 2026, for DeepSeek-V4-Flash-Vision-Exp."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is DeepSeek V4 Pro 0813 (#42)."],["scores.headroom","pass","The best is 71.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dscodebench","name":"DSCodeBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","3 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Qwen2.5-Coder-32B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 22.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dsi-bench","name":"DSI-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries) and system cards (1 score)."],["scores.models","pass","11 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 61.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 40 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dtap-direct-misuse","name":"DTap Direct Misuse","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries) and system cards (1 score)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on decodingtrust-agent.com: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dtap-prompt-injection","name":"DTap Prompt Injection","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries) and system cards (1 score)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on decodingtrust-agent.com: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dtbench","name":"DTBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (203 entries)."],["scores.models","pass","187 models we track, plus 16 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 99.2%, past 85% since June 2025."],["import.importer","pass","We read the board on conceptualreasoning.ai: 252 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dtu-multi-view-stereo","name":"DTU Multi-View Stereo","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dual-exam","name":"Dual Exam","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dual-arm-sim","name":"Dual-Arm Sim","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on micro1.ai: 11 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dualchem","name":"DualChem","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (33 entries)."],["scores.models","pass","32 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 67.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on scale.com: 32 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dualentry-accounting-ai-benchmark","name":"DualEntry Accounting AI Benchmark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","29 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 91.1%, past 85% since August 2026."],["import.importer","pass","We read the board on dualentry.com: 29 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dubai-cc","name":"Dubai-CC","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dubesor-ai-chess","name":"Dubesor AI Chess Leaderboard","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (306 entries)."],["scores.models","pass","250 models we track, plus 56 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on dubesor.de: 713 rows for models we track and 56 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dubesor-lichess-puzzle-rating","name":"Dubesor Lichess Puzzle Rating","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (66 entries)."],["scores.models","pass","62 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on dubesor.de: 225 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dude","name":"DUDE","summary":{"pass":8,"warn":1,"fail":1,"info":2,"text":"8 of 10 checks pass, 1 warning, 1 problem"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","4 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#174)."],["scores.headroom","pass","The best is 82.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 12 entries at the last read, none a model we track."],["import.read","warn","November 11th, 2026, 10 days before the data’s date."],["import.failed","fail","The read on November 19th, 2026 failed with “Command failed: curl --fail --silent --show-error --max-time 40 --user-agent benchmark-almanac (+https://gith…”. Pages show the rows from November 11th, 2026."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"duellab-gamebench-2","name":"DuelLab GameBench 2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (67 entries)."],["scores.models","pass","56 models we track, plus 11 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on benchmarks.duellab.org: 110 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"duetbench","name":"DuetBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on fireworks.ai: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"duidelijke-taal","name":"Duidelijke Taal","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dumatebench","name":"DuMateBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on dumatebench.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"duobench","name":"DuoBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for π0.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 10.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on duobench.github.io: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"duplex-mpe","name":"Duplex-MPE","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","2 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for MiniCPM-o 4.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 72.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on step-out.github.io: 4 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"duplexledger","name":"DuplexLedger","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","2 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for MiniCPM-o 4.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 40.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"duplicate-side-effect-desk","name":"Duplicate Side Effect Desk","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Claude Haiku 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 4.5 (#81)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dutch-book-bench","name":"Dutch Book Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","17 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Sonnet 5.5 (#9)."],["scores.headroom","info","Saturated. The best is 89.2%, past 85% since September 2026."],["import.importer","pass","We read the board on kaggle.com: 17 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dutch-book-reviews-dataset","name":"Dutch Book Reviews Dataset","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (352 entries)."],["scores.models","pass","105 models we track, plus 247 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Nemotron 3 Nano 4B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3-235B-A22B (#131)."],["scores.headroom","info","Saturated. The best is 92.5%, past 85% since July 2023."],["import.importer","pass","We read the board on EuroEval: 118 rows for models we track and 247 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dutch-cola","name":"Dutch CoLA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (257 entries)."],["scores.models","pass","21 models we track, plus 236 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Olmo 3.1 32B Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is gpt-oss-20b (#143)."],["scores.headroom","pass","The best is 49.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on euroeval.com: 24 rows for models we track and 236 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dutch-dpa-voting-advice-2025","name":"Dutch DPA national voting advice audit","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dutch-model-benchmark","name":"Dutch Model Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","14 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 14 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dycodeeval","name":"DyCodeEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Llama 3.2 1B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-8B (#203)."],["scores.headroom","pass","The best is 39%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dynabench-squad","name":"Dynabench SQuAD","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dynamath","name":"DynaMath","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries), launch posts (17 scores) and system cards (12 scores)."],["scores.models","pass","34 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.6 Plus (#75)."],["scores.headroom","info","Saturated. The best is 88%, past 85% since February 2026."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dynamic-programming-m8ngotree","name":"Dynamic Programming (m8ngotree)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dynamic-superb-phase-2","name":"Dynamic-SUPERB Phase-2","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dau-or-die-vietnamese-diacritic-restoration","name":"Dấu or Die: Vietnamese Diacritic Restoration","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Haiku 5.5 (#20)."],["scores.headroom","info","Saturated. The best is 97.3%, past 85% since June 2025."],["import.importer","pass","We read the board on kaggle.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"e-bench-tencent","name":"E-Bench (Tencent)","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries) and launch posts (2 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 80.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"e-bench-code-tencent","name":"E-Bench-Code (Tencent)","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries) and launch posts (1 score)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 83.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"e-commerce-bench","name":"E-Commerce Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 21 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"psychosmiley-erp-leaderboard","name":"E/RP Leaderboard by PsychoSmiley","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from June 2026, for Nemotron 3 Ultra."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on psychosmiley.github.io: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"e2e-swe","name":"E2E-SWE","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"earnings-21","name":"Earnings-21","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"earnings-22","name":"Earnings-22","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"earth-bench","name":"Earth-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","10 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash. That’s expected for a test replaced by Earth-Bench V2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 66%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 20 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"earth-bench-2","name":"Earth-Bench V2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (48 entries)."],["scores.models","pass","47 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Axis Spatial: 91 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"earth-bench-pro","name":"Earth-Bench-Pro","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (21 entries)."],["scores.models","pass","21 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 20 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"earthrecord","name":"EarthRecord","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 85.4%, past 85% since September 2026."],["import.importer","pass","We read the board on ranranrannervous.github.io: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"earthverse","name":"EarthVerse","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","17 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 85%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cuizhiq.github.io: 18 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ebench","name":"EBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ebm-nlp","name":"EBM-NLP","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ebm-nlpmod","name":"EBM-NLPmod","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ebr-bench","name":"EBR-bench","summary":{"pass":7,"warn":1,"fail":0,"info":2,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from Epoch AI’s runs (34 models)."],["scores.models","pass","34 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 92.4%, past 85% since November 2026."],["import.importer","warn","The record links a board on epoch.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"echo-prediction-leaderboard-unipat","name":"Echo General AI Prediction Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on echo.unipat.ai: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"echochain-labelbox","name":"EchoChain","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for gpt-realtime."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 44%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labelbox.com: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"echr-binary-violation-classification","name":"ECHR Binary Violation Classification","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"echr-argument-mining-poudyal","name":"ECHR Legal Corpus for Argument Mining","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 1 entry at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eclektic","name":"ECLeKTic","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (35 entries), launch posts (1 score) and system cards (7 scores)."],["scores.models","pass","37 models we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 56.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 35 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 5 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"economic-dispatch","name":"Economic Dispatch","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"economics-arena","name":"Economics Arena for Large Language Models","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Claude 2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 2 (#192)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"edb-bio-edison","name":"EDB-Bio","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"edgebench","name":"EdgeBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 30 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"edinet-bench","name":"EDINET-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","8 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Kimi K2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on sakana.ai: 24 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"edis","name":"EDIS","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (43 entries)."],["scores.models","pass","43 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MTEB: 43 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eduart","name":"EduArt","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","10 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 82.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 10 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"edubench-ru","name":"EduBench-RU","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","21 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 21 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"education-bench-bytedance","name":"Education (ByteDance Seed in-house)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Seed1.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"educlaw-bench","name":"EduClaw-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eduexplain","name":"EduExplain","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"edufair-bench","name":"EduFair-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eduvideobench","name":"EduVideoBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on anonymous.4open.science: 6 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eebench","name":"EEBench","summary":{"pass":10,"warn":2,"fail":0,"info":0,"text":"10 of 12 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (34 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","34 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 83.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on eebench.org: 36 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"effective-fps","name":"Effective FPS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"egi-bench","name":"EGI-BENCH","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","12 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ego-exo4d","name":"Ego-Exo4D","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ego-mc-bench","name":"Ego-MC-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","8 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on apratimbh.github.io: 10 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ego2act","name":"Ego2Act","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Cosmos3-Nano."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ego2act.github.io: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ego2flow","name":"Ego2Flow","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ego3d-bench","name":"Ego3D-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries) and launch posts (1 score)."],["scores.models","pass","12 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.7-Plus (#79)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on raw.githubusercontent.com: 22 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ego4d-goal-step","name":"Ego4D Goal-Step","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on eval.ai: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ego4d-state-change-object-detection","name":"Ego4D State Change Object Detection","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on eval.ai: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"egoblind","name":"EgoBlind","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","7 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#112)."],["scores.headroom","pass","The best is 59.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"egocross","name":"EgoCross","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","14 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on egocross-benchmark.github.io: 14 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"egohome","name":"EgoHome","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"egolife-with-memory","name":"EgoLife (with memory)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 80.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"egoper-error-detection","name":"EgoPER Error Detection","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"egoprocel","name":"EgoProceL","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"egoprocevqa","name":"EgoProceVQA","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","6 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 51.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on z1oong.github.io: 11 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"egoschema","name":"EgoSchema","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries), launch posts (12 scores) and system cards (5 scores)."],["scores.models","pass","12 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Amazon Nova Premier."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok-3 mini (#127)."],["scores.headroom","pass","The best is 76.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"egotempo","name":"EgoTempo","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries), launch posts (3 scores) and system cards (4 scores)."],["scores.models","pass","11 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Seed2.0 Lite (0428)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 71.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"egotom","name":"EgoToM","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"egyhellaswag","name":"EgyHellaSwag","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","5 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-8B (#154)."],["scores.headroom","pass","The best is 40.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"egymmlu","name":"EgyMMLU","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ehrnoteqa","name":"EHRNoteQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","3 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in December 2023, for Mistral-7B-Instruct-v0.2. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","info","Saturated. The best is 96.2%, past 85% since June 2023."],["import.importer","pass","We read the board on proceedings.neurips.cc: 6 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ehrqa-google","name":"EHRQA (Google)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ehrshot","name":"EHRSHOT","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ehrshot.stanford.edu: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ehrsql","name":"EHRSQL","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eicat-impact-classification","name":"EICAT impact classification","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-8B (#203)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 11 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eifbench","name":"EIFBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Seed1.8."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 48.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ejbench","name":"EJBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eki-eesti-mang","name":"EKI Eesti mäng","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (101 entries)."],["scores.models","pass","96 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 90%, past 85% since February 2026."],["import.importer","pass","We read the board on xn--mdupuu-pxaa.eki.ee: 96 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eki-malumang","name":"EKI Mälumäng","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (93 entries)."],["scores.models","pass","89 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 50.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on xn--mdupuu-pxaa.eki.ee: 89 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eki-propaganda-resistance","name":"EKI Propaganda Resistance","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (101 entries)."],["scores.models","pass","96 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on xn--mdupuu-pxaa.eki.ee: 96 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ekw-turing","name":"EKWbench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 74.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on bench.turing.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"elbench","name":"ELBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","7 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 7 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"elcardiocc-2025","name":"ELCardioCC 2025","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on participants-area.bioasq.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"election-integrity-evaluations-anthropic","name":"Election integrity evaluations (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"election-questions-anthropic","name":"Election Questions (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dosimeter-german-election-day-2026","name":"Election-Day AI Poisoning in Germany","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on dosimeteraudit.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"demos-electoral-hallucinations-scotland","name":"Electoral Hallucinations","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on demos.co.uk: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"elephantbench","name":"ElephantBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","29 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 52.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tencent.github.io: 29 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eleusis-nph4rd","name":"Eleusis","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-6 Sol (#13)."],["scores.headroom","pass","The best is 82.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on nphard.io: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-eli5-arena","name":"ELI5 Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 20 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"elimination-game","name":"Elimination Game","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (57 entries)."],["scores.models","pass","56 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 60 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ellipse-holistic-essay-scoring","name":"ELLIPSE Holistic Essay Scoring","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"emailbench","name":"EmailBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 33.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodied-agent-arena","name":"Embodied Agent Arena","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodied-agent-interface","name":"Embodied Agent Interface","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","13 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for o1-mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on embodied-agent-interface.github.io: 13 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodied-first-exam","name":"Embodied First Exam","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","5 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 49.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on embodied-first-exam.ai: 10 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodied-over-refusal-benchmark","name":"Embodied Over-Refusal Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodiedbench","name":"EmbodiedBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","21 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for InternVL3-38B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 50.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on embodiedbench.github.io: 21 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodiedbench-eb-alfred","name":"EmbodiedBench EB-ALFRED","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","17 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for InternVL3-38B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 67.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on embodiedbench.github.io: 17 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodiedbench-eb-habitat","name":"EmbodiedBench EB-Habitat","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","17 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for InternVL3-38B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 58.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on embodiedbench.github.io: 17 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodiedbench-eb-manipulation","name":"EmbodiedBench EB-Manipulation","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","17 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for InternVL3-38B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 28.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on embodiedbench.github.io: 17 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodiedbench-eb-navigation","name":"EmbodiedBench EB-Navigation","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","17 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for InternVL3-38B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 53.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on embodiedbench.github.io: 17 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodiedclbench","name":"EmbodiedCLBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Terra (#18)."],["scores.headroom","pass","The best is 52%, under 85%, so it still separates models."],["import.importer","pass","We read the board on pku-value-lab.github.io: 24 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodiedclue-vla-planning","name":"EmbodiedCLUE-VLA (planning)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","4 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 4 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodiedclue-vla-tabletop-tasks","name":"EmbodiedCLUE-VLA (tabletop tasks)","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","2 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GR00T N1.6."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 2 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodiedclue-vla-robotwin","name":"EmbodiedCLUE-VLA RoboTwin Evaluation","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on superclueai.com: 17 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodiedclue-world","name":"EmbodiedCLUE-World","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodiedclue-world-2-0","name":"EmbodiedCLUE-World 2.0","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for LingBot-Video-MoE."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodiedintentbench","name":"EmbodiedIntentBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodiedlab","name":"EmbodiedLab","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 41.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embodiedverse-bench","name":"EmbodiedVerse-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","17 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 17 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"embspatialbench","name":"EmbSpatialBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries), launch posts (14 scores) and system cards (5 scores)."],["scores.models","pass","19 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.5 397B-A17B (#78)."],["scores.headroom","info","Saturated. The best is 85.4%, past 85% since March 2026."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"emdb","name":"EMDB","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eth-ait.github.io: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"emergent-collusion-lech-mazur","name":"Emergent Price-Fixing by LLM Auction Agents","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","12 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Grok 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 12 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"emma-multimodal-reasoning","name":"EMMA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries), launch posts (1 score) and system cards (3 scores)."],["scores.models","pass","10 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash Thinking (#148)."],["scores.headroom","pass","The best is 79.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on emma-benchmark.github.io: 13 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"emobench-m","name":"EmoBench-M","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","16 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 70.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 16 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"emotional-attachment-eval-google","name":"Emotional Attachment evaluation","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","pass","The best is 76%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"emotiontalk","name":"EmotionTalk","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","1 model we track, plus 25 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in June 2023, for chatglm2-6b."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is chatglm2-6b (#244)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 25 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"empatheticdialogues","name":"EmpatheticDialogues","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"employment-law-legal-agent-eval","name":"Employment Law Legal Agent Eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"egocentric-vqa-encord","name":"Encord Egocentric VQA Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","11 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 76.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on encord.com: 11 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"encqa","name":"EncQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"encyclo-k","name":"Encyclo-K","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (49 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","30 models we track, plus 20 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Seed2.0 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 65.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on encyclo-k.github.io: 38 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"encyclopedic-vqa","name":"Encyclopedic VQA","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on openaccess.thecvf.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"end-to-end-swe-task-benchmark-windsurf","name":"End-To-End SWE Task Benchmark (Windsurf)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 18 months ago, in May 2025, for SWE-1. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"endovis-2018-robotic-scene-segmentation","name":"EndoVis 2018 Robotic Scene Segmentation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"endovis18-vqa","name":"EndoVis18-VQA","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enem-challenge","name":"ENEM Challenge","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"engiworld","name":"EngiWorld","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 44.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on engiworld.github.io: 7 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt26-icelandic-syntactic-divergence","name":"English to Icelandic Test Suite for Syntactic Divergence","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (27 entries)."],["scores.models","pass","13 models we track, plus 14 other entries on its leaderboard."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Command A+. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 93.4%, past 85% since February 2026."],["import.importer","pass","We read the board on aclanthology.org: 13 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enigmaeval","name":"EnigmaEval","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 39.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterprise-generative-eval-cohere","name":"Enterprise generative evaluation (Cohere)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Cohere Command A. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 94.2%, past 85% since April 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterprise-pii-patronus","name":"Enterprise PII","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterprise-rag-eval-cohere","name":"Enterprise RAG evaluation (Cohere)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Cohere Command A. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"creative-writing-patronus","name":"Enterprise Scenarios Creative Writing","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"customer-support-dialogue-patronus","name":"Enterprise Scenarios Customer Support Dialogue","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterprise-scenarios-patronus","name":"Enterprise Scenarios Leaderboard","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toxicity-patronus","name":"Enterprise Scenarios Toxicity","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterprise-task-pareto-frontier-glean","name":"Enterprise Task Pareto Frontier (Glean)","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","warn","The record links a board on glean.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterprise-vendor-eval","name":"Enterprise Vendor Eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterprise-bench-l1-l2","name":"Enterprise-Bench L1-L2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","10 models we track, plus 24 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 82.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on hub.harborframework.com: 34 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterprise-bench-l1-l2-v2","name":"Enterprise-Bench L1-L2 v2","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 65.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on hub.harborframework.com: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterprisebench-corecraft","name":"EnterpriseBench: CoreCraft","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (55 entries) and launch posts (1 score)."],["scores.models","pass","53 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Small."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 77.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on surgehq.ai: 53 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterpriseclawbench","name":"EnterpriseClawBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 66.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on frontisai.github.io: 32 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterpriseops-gym","name":"EnterpriseOps-Gym","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries), launch posts (4 scores) and system cards (1 score)."],["scores.models","pass","16 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 46.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 12 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterpriseops-gym-1-1","name":"EnterpriseOps-Gym 1.1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterpriserag-bench","name":"EnterpriseRAG-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (43 entries)."],["scores.models","pass","2 models we track, plus 41 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","info","Saturated. The best is 91.5%, past 85% since July 2026."],["import.importer","pass","We read the board on huggingface.co: 2 rows for models we track and 41 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"entiresongs-qwen","name":"EntireSongs","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Qwen3-ASR-1.7B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"entity-deduction-arena","name":"Entity-Deduction Arena","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enviroexam","name":"EnviroExam","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (31 entries)."],["scores.models","pass","10 models we track, plus 21 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Gemma 7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 7B (#211)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 20 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"envisioning-signals-benchmark","name":"Envisioning Signals Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (47 entries)."],["scores.models","pass","42 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","5 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on signals.envisioning.com: 42 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"envsimbench","name":"EnvSimBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for MiniMax-M2.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 42.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"epibench","name":"EpiBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 39.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 42 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"epic-kitchens-visor-hand-object-segmentation","name":"EPIC-KITCHENS VISOR Hand Object Segmentation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on codalab.lisn.upsaclay.fr: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"epic-kitchens-visor-semi-supervised-vos","name":"EPIC-KITCHENS VISOR Semi-Supervised Video Object Segmentation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on epic-kitchens.github.io: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"epic-kitchens-100-action-anticipation","name":"EPIC-KITCHENS-100 action anticipation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on codalab.lisn.upsaclay.fr: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"epic-tent-o","name":"Epic-tent-O","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"epistemic-bias-xai","name":"Epistemic Bias (xAI)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Grok 4.5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.5 (#47)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"epoch-no-cot-serial-steps","name":"Epoch AI no-CoT serial reasoning steps","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"epoch-automation-reports","name":"Epoch Automation Reports","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 81.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on epoch.ai: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"epoch-capabilities-index","name":"Epoch Capabilities Index","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","warn","The record links a board on epoch.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eprstmt","name":"EPRSTMT","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eq-bench-3","name":"EQ-Bench 3","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (82 entries) and launch posts (1 score)."],["scores.models","pass","79 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on eqbench.com: 79 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eq-bench-4","name":"EQ-Bench 4","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","32 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on eqbench.com: 32 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eq-bench-2","name":"EQ-Bench v2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (172 entries)."],["scores.models","pass","86 models we track, plus 86 other entries on its leaderboard."],["scores.recent","info","The newest came 20 months ago, in March 2025, for GPT-4o. That’s expected for a test replaced by EQ-Bench 3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eqbench.com: 86 rows for models we track and 86 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"era-by-eon","name":"Era by Eon","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 76.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"erp-bench-agentic-labs","name":"ERP-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 43.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on erpbench.ai: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"erpbench-100","name":"ERPBench-100","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"erqa","name":"ERQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries), launch posts (28 scores) and system cards (15 scores)."],["scores.models","pass","54 models we track, plus 22 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 78.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on embodied-arena.com: 18 rows for models we track and 22 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"err-estonian-language-and-cultural-history","name":"ERR Estonian language and cultural history questions","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on e.infogram.com: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"errnews","name":"ERRnews broadcast summarization","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on bjmc.lu.lv: 7 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"esbbq","name":"EsBBQ","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (34 entries)."],["scores.models","pass","11 models we track, plus 23 other entries on its leaderboard."],["scores.recent","info","The newest came 20 months ago, in March 2025, for Gemma 3 12B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.1-8B (#203)."],["scores.headroom","info","Saturated. The best is 85%, past 85% since September 2024."],["import.importer","pass","We read the board on arxiv.org: 11 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"esc-50","name":"ESC-50","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","32 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 32 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"escalation-router","name":"Escalation Router","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"esci-shopping-queries-dataset","name":"ESCI Shopping Queries Dataset","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aicrowd.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"escola","name":"EsCoLA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"esi-bench","name":"ESI-Bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 70%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"estner","name":"EstNER","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 1 entry at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"estonian-ai-barometer","name":"Estonian AI Barometer","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (55 entries)."],["scores.models","pass","25 models we track, plus 30 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on baromeeter.tartunlp.ai: 25 rows for models we track and 30 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"estonian-valence-corpus","name":"Estonian Valence Corpus","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (193 entries)."],["scores.models","pass","5 models we track, plus 188 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 59.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MTEB: 5 rows for models we track and 188 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eth3d","name":"ETH3D","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ether0-benchmark-futurehouse","name":"ether0 benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ethics-hendrycks","name":"ETHICS","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on raw.githubusercontent.com: 6 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eticor","name":"EtiCor","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in March 2023, for Falcon-40B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Falcon-40B (#230)."],["scores.headroom","pass","The best is 63%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"etva","name":"ETVA","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"euchre-bench","name":"euchre-bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on fewworddotrick.com: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eur-lex-sum","name":"EUR-Lex-Sum","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eureka-bench-openai","name":"EUREKA-Bench (OpenAI)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eurekabench","name":"EurekaBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eurekabench.github.io: 7 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"euroeval","name":"EuroEval","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (143 entries)."],["scores.models","pass","75 models we track, plus 68 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on euroeval.com: 99 rows for models we track and 68 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"euroeval-err-news","name":"EuroEval ERRNews","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (199 entries)."],["scores.models","pass","76 models we track, plus 123 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on euroeval.com: 89 rows for models we track and 123 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"faroese-grammatical-error-correction-euroeval","name":"EuroEval Faroese Grammatical Error Correction","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (241 entries)."],["scores.models","pass","71 models we track, plus 170 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for LFM2.5-1.2B-Thinking."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is QwQ-32B (#137)."],["scores.headroom","pass","The best is 16.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on euroeval.com: 75 rows for models we track and 170 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"european-llm-leaderboard-opengpt-x","name":"European LLM Leaderboard (OpenGPT-X)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (52 entries)."],["scores.models","pass","13 models we track, plus 39 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Mixtral 8x7B (#199)."],["scores.headroom","pass","The best is 56.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 26 rows for models we track and 39 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"european-parliament-soteu-2010-sentiment","name":"European Parliament 2010 State of the Union sentiment validation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dri-european-parliament-chatbot-audit-2024","name":"European Parliament election chatbot audit","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on democracyreporting.s3.eu-central-1.amazonaws.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eurosat","name":"EuroSAT","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (58 entries)."],["scores.models","pass","1 model we track, plus 57 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 83.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MTEB: 1 row for models we track and 57 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eusexams","name":"EusExams","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (69 entries)."],["scores.models","pass","24 models we track, plus 45 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-8B (#154)."],["scores.headroom","pass","The best is 51.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on La Leaderboard: 24 rows for models we track and 45 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eusproficiency","name":"EusProficiency","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","9 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-3.5 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","pass","The best is 56.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eusreading","name":"EusReading","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (69 entries)."],["scores.models","pass","24 models we track, plus 45 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-8B (#154)."],["scores.headroom","pass","The best is 62.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on La Leaderboard: 24 rows for models we track and 45 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eustrivia","name":"EusTrivia","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (69 entries)."],["scores.models","pass","24 models we track, plus 45 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-8B (#154)."],["scores.headroom","pass","The best is 51%, under 85%, so it still separates models."],["import.importer","pass","We read the board on La Leaderboard: 24 rows for models we track and 45 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eva-bench-servicenow","name":"EVA-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (24 entries) and launch posts (1 score)."],["scores.models","pass","11 models we track, plus 13 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Grok Voice 3.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on servicenow.github.io: 12 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evacun-2025","name":"EvaCun 2025","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on digitalpasts.github.io: 3 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evahan-2023","name":"EvaHan 2023","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evahan-2024","name":"EvaHan 2024","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 1 entry at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evahan-2025","name":"EvaHan 2025","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evahan-2026","name":"EvaHan 2026","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evalatin-2024","name":"EvaLatin 2024","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evalatin-2026","name":"EvaLatin 2026","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 77.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on circse.github.io: 2 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evalatro-2","name":"Evalatro v2","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (62 entries)."],["scores.models","pass","28 models we track, plus 34 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on evalatro.dev: 29 rows for models we track and 34 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evalcheck-llm","name":"EvalCheck-LLM","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.6 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.6 Flash (#45)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since July 2025."],["import.importer","pass","We read the board on kaggle.com: 3 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evalconvolearn","name":"EvalConvoLearn","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evalcrafter","name":"EvalCrafter","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on evalcrafter.github.io: 14 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evaldetectbench","name":"EvalDetectBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GLM-5.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on evaldetectbench.com: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evalearn","name":"EvaLearn","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evalita-llm","name":"Evalita-LLM","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (48 entries)."],["scores.models","pass","27 models we track, plus 21 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Qwen3-Next-80B-A3B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is gpt-oss-20b (#143)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 49 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evalplus","name":"EvalPlus","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (5 scores)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Qwen3-Coder-Next. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GLM-5 (#89)."],["scores.headroom","info","Saturated. The best is 87%, past 85% since February 2026."],["import.importer","warn","The record links a board on evalplus.github.io that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-teen-safety-72-scenarios","name":"Evaluating AI Safety in Teen Conversations","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","7 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Muse Spark 1.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 5 (#35)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on vals.ai: 7 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evaluation-awareness-discriminability-anthropic","name":"Evaluation awareness discriminability (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mcbain-suicide-risk-assessment","name":"Evaluation of Alignment Between Large Language Models and Expert Clinicians in Suicide Risk Assessment","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eotgc","name":"Evaluation of Essay On-Topic Graded Comments","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evalverse","name":"EvalVerse","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evaomni","name":"EvaOmni","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","22 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","pass","The best is 79.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on xiaoxing2001.github.io: 23 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"event-based-segmentation-dataset","name":"Event-based Segmentation Dataset","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eventhallusion","name":"EventHallusion","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","1 model we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for GPT-4V."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 76.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evf-bench","name":"EVF-Bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","1 model we track, plus 9 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evidence-inference-2","name":"Evidence Inference 2.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evidence-sufficiency-evaluation","name":"Evidence Sufficiency Evaluation","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on kaggle.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evmbench","name":"EVMbench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 51%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openai.com: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evoarena","name":"EvoArena","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 53.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aiden0526.github.io: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evoarena-personamem-evo","name":"EvoArena PersonaMem-Evo","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aiden0526.github.io: 16 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evoarena-swe-chain-evo","name":"EvoArena SWE-Chain-Evo","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aiden0526.github.io: 16 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evoarena-terminal-bench-evo","name":"EvoArena Terminal-Bench-Evo","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 65.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aiden0526.github.io: 16 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evocode-bench","name":"EvoCode-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash. That’s expected for a test replaced by EvoCode-Bench v2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 54%, under 85%, so it still separates models."],["import.importer","pass","We read the board on unipat.ai: 13 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evocode-bench-v2","name":"EvoCode-Bench v2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 59.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on unipat.ai: 13 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evoeval","name":"EvoEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (51 entries)."],["scores.models","pass","19 models we track, plus 32 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for GPT-4 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#172)."],["scores.headroom","pass","The best is 68.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on evo-eval.github.io: 19 rows for models we track and 32 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evopolicygym","name":"EvoPolicyGym","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evve","name":"EVVE","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on openaccess.thecvf.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ewmbench","name":"EWMBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ewok","name":"EWoK","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (36 entries)."],["scores.models","pass","25 models we track, plus 11 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in October 2024, for Claude 3.5 Haiku. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","info","Saturated. The best is 91.2%, past 85% since February 2024."],["import.importer","pass","We read the board on arxiv.org: 25 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"exact-words","name":"Exact Words","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","12 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Sonnet 5.5 (#9)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since May 2025."],["import.importer","pass","We read the board on kaggle.com: 12 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"exam-et","name":"Exam-et","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"exams-multilingual-qa","name":"EXAMS","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"exams-v","name":"EXAMS-V","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for Gemini 1.0 Pro Vision."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 44%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-excel-modeling-benchmark","name":"Excel Modeling Benchmark","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (76 entries)."],["scores.models","pass","76 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 76.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 77 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-excel-modeling-benchmark-1-1","name":"Excel Modeling Benchmark v1.1","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (47 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","47 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 77.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 46 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"excel-fix-bench-tencent","name":"ExcelFixBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Tencent Hy4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek V4.1 Flash 1017 (#23)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"exeworldbench","name":"ExeWorldBench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"exfuntube","name":"ExFunTube","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"exp-bench","name":"EXP-Bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 0.50%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"expected-cyber-refusals-openai","name":"Expected Cyber Refusals","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 99.4%, past 85% since November 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"expert-comparisons-biothreat-openai","name":"Expert comparisons on biothreat information","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 22 months ago, in January 2025, for o3-mini. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dellavigna-pope-real-effort-forecasts","name":"Expert Forecasts of Real-Effort Experiment","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"attack-planning-red-teaming-openai","name":"Expert Red Teaming for Violent Attack Planning","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 65.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"expert-vulnerability-discovery-and-exploit-development-anthropic","name":"Expert vulnerability discovery and exploit development evaluations (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in March 2024, for Claude 3 Opus. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"expert-swe-openai","name":"Expert-SWE (OpenAI)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6 Cyber."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 83%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"expertcrowd-swe","name":"ExpertCrowd-SWE","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 67.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"experteval-unipat","name":"ExpertEval","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","7 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Qwen 3.6 Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 66.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on unipat.ai: 7 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"expertlongbench","name":"ExpertLongBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","10 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 33.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 10 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"exploitbench","name":"ExploitBench","summary":{"pass":9,"warn":0,"fail":1,"info":2,"text":"9 of 10 checks pass, 1 problem"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries), launch posts (8 scores) and system cards (22 scores)."],["scores.models","pass","33 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2026."],["import.importer","pass","We read the board on exploitbench.ai: 20 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","fail","The read on November 19th, 2026 failed with “403 https://exploitbench.ai/”. Pages show the rows from November 18th, 2026."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"exploitbench-internal-port-openai","name":"ExploitBench - Internal Port","summary":{"pass":7,"warn":0,"fail":0,"info":3,"text":"All 7 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (3 scores) and system cards (7 scores)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 53.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"exploitgym","name":"ExploitGym","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries), launch posts (8 scores) and system cards (15 scores)."],["scores.models","pass","25 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Carbon."],["scores.frontier","pass","5 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 54.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cybergym.io: 27 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 4 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"explorationbench-tencent","name":"ExplorationBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 89%, past 85% since July 2026."],["import.importer","pass","We read the board on explorationbench.com: 10 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"expunations","name":"ExPUNations","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nyt-connections-extended","name":"Extended NYT Connections","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (110 entries)."],["scores.models","pass","109 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 99.3%, past 85% since November 2025."],["import.importer","pass","We read the board on github.com: 143 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"external-pilot-use-failure-modes-anthropic","name":"External pilot use failure modes (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"red-team-jailbreaks-openai","name":"External red team jailbreaks","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 2 years ago, in September 2024, for o1-mini. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1-mini (#142)."],["scores.headroom","pass","The best is 44%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"extract-bench-contextual","name":"Extract Bench (Contextual AI)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"extractbench","name":"ExtractBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (56 entries)."],["scores.models","pass","34 models we track, plus 22 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 95.3%, past 85% since February 2026."],["import.importer","pass","We read the board on github.com: 34 rows for models we track and 22 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"extractbench-pro","name":"ExtractBench Pro","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","7 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 58%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"extremedrive","name":"ExtremeDrive","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eyepacs-dr-google","name":"EyePACS (diabetic retinopathy grading)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"factorio-learning-environment","name":"Factorio Learning Environment","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Gemini 2.0 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on jackhopkins.github.io: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"factorio-play","name":"Factorio Play","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"code-review-benchmark-factory","name":"Factory Code Review Benchmark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 60.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on factory.com: 13 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"factorybench-forgis","name":"FactoryBench (Forgis Labs)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 4.6 (#57)."],["scores.headroom","pass","The best is 36%, under 85%, so it still separates models."],["import.importer","pass","We read the board on forgis.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"factorybench-100","name":"FactoryBench-100","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Luna (#36)."],["scores.headroom","pass","The best is 76.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on blobfish.ai: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"facts-about-singapore","name":"Facts about Singapore","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"facts-benchmark-suite","name":"FACTS Benchmark Suite","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries), launch posts (4 scores) and system cards (1 score)."],["scores.models","pass","26 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","7 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 78.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 23 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"facts-grounding","name":"FACTS Grounding","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (35 entries), launch posts (10 scores) and system cards (11 scores)."],["scores.models","pass","42 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 88.5%, past 85% since February 2025."],["import.importer","pass","We read the board on kaggle.com: 34 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 7 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"facts-grounding-v2","name":"FACTS Grounding v2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","26 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","7 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 84.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 26 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"facts-multimodal","name":"FACTS Multimodal","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","23 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","7 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 60.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 23 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"facts-parametric","name":"FACTS Parametric","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (46 entries) and launch posts (1 score)."],["scores.models","pass","47 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 83.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 47 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"facts-parametric-v2","name":"FACTS Parametric V2","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","28 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","7 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 83.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 28 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"facts-search","name":"FACTS Search","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","31 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. That’s expected for a test replaced by FACTS Search V2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 85.6%, past 85% since February 2026."],["import.importer","pass","We read the board on kaggle.com: 31 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"facts-search-v2","name":"FACTS Search V2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","22 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 89.9%, past 85% since July 2026."],["import.importer","pass","We read the board on kaggle.com: 22 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"factscore","name":"FActScore","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries), launch posts (5 scores) and system cards (4 scores)."],["scores.models","pass","3 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Seed2.0 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Dolly 2.0-12b (#256)."],["scores.headroom","pass","The best is 71.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"factuality-heavy-prompts-openai","name":"Factuality Heavy prompts","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Sol (#13)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fad-music-listening-test","name":"FAD Music Listening Test","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"failsafeqa","name":"FailSafeQA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (24 entries)."],["scores.models","pass","15 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 15 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"failuresensoriq","name":"FailureSensorIQ","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","25 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 69.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 25 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"faintasy-football","name":"fAIntasy Football","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on faintasy.football: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fairface","name":"FairFace","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fairscreen-testing-llm-hiring-decisions","name":"FairScreen: Testing LLM Hiring Decisions","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Gemma 4 31B IT. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 Nano (#90)."],["scores.headroom","info","Saturated. The best is 98.4%, past 85% since April 2026."],["import.importer","pass","We read the board on kaggle.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fairstress","name":"FairStress","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"faithdial","name":"FaithDial","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-faithfulness-moonshot","name":"Faithfulness (Moonshot)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Saturated. The best is 85.5%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"faithjudge","name":"FaithJudge","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (37 entries) and system cards (1 score)."],["scores.models","pass","38 models we track."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Kimi K2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 46 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fake-or-real","name":"Fake-or-Real","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fake-br-corpus","name":"Fake.Br Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fakerecogna","name":"FakeRecogna","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"faketruebr","name":"FakeTrueBR","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on sol.sbc.org.br: 1 entry at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"false-premise-questions-anthropic","name":"False-premise questions (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"falseclaimbench-xai","name":"FalseClaimBench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Grok 4.5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.5 (#47)."],["scores.headroom","pass","The best is 84%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fantasyfootballbench","name":"FantasyFootballBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ffasr-leaderboard","name":"Far-Field ASR Leaderboard","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (61 entries)."],["scores.models","pass","7 models we track, plus 54 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for NVIDIA Nemotron 3.5 Transcribe 30B-A4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on huggingface.co: 7 rows for models we track and 54 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"faroese-grammatical-correctness","name":"Faroese Grammatical Correctness","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 90.9%, past 85% since March 2025."],["import.importer","pass","We read the board on aclanthology.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"faroese-metaphorical-explanations","name":"Faroese Metaphorical Explanations","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 88.4%, past 85% since August 2025."],["import.importer","pass","We read the board on aclanthology.org: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"faroese-semantic-textual-similarity","name":"Faroese Semantic Textual Similarity","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (193 entries)."],["scores.models","pass","4 models we track, plus 189 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MTEB: 4 rows for models we track and 189 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"farsick","name":"FarSick","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (50 entries)."],["scores.models","pass","3 models we track, plus 47 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Qwen3-Embedding-0.6B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mteb-leaderboard.hf.space: 3 rows for models we track and 47 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fashion-iq","name":"Fashion IQ","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (44 entries)."],["scores.models","pass","44 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on MTEB: 44 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fastapi-competency","name":"FastAPI Competency","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fate-h","name":"FATE-H","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","8 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Leanstral 1.5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","info","Saturated. The best is 87%, past 85% since July 2026."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fate-x","name":"FATE-X","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","8 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Leanstral 1.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 34%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fda-vaers-annotations","name":"FDA and CDC VAERS clinical and temporal annotations","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fda-information-extraction-based","name":"FDA Information Extraction (Based)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fde-gym","name":"FDE-Gym","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"feabench","name":"FEABench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"feateng","name":"FeatEng","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (22 entries)."],["scores.models","pass","16 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Gemini 1.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","pass","The best is 11.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 17 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"featurebench","name":"FeatureBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","5 models we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for MiniMax-M2.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 60.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 20 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"featurebench-lite","name":"FeatureBench Lite","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","5 models we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for MiniMax-M2.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 60.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 20 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fechamento-br","name":"Fechamento BR","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"feedback-prize-english-language-learning","name":"Feedback Prize: English Language Learning","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2,655 entries)."],["scores.models","pass","1 model we track, plus 2,654 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for o3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 1 row for models we track and 2,654 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"feedback-prize-evaluating-student-writing","name":"Feedback Prize: Evaluating Student Writing","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2,058 entries)."],["scores.models","pass","2,058 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 2,058 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"feedback-prize-predicting-effective-arguments","name":"Feedback Prize: Predicting Effective Arguments","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1,558 entries)."],["scores.models","pass","1 model we track, plus 1,557 other entries on its leaderboard."],["scores.recent","info","The newest came 4 years ago, in August 2022, for Sora. That’s expected for a test replaced by Feedback Prize: English Language Learning."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 1 row for models we track and 1,557 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"feedbackqa","name":"FeedbackQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on MTEB: 15 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"feishu-office-v1","name":"Feishu Office v1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fem-bench","name":"FEM-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Claude Opus 4.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Saturated. The best is 87.9%, past 85% since November 2025."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fenbench","name":"FenBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fermat-last-theorem-lean-anthropic","name":"Fermat's Last Theorem in Lean (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 35%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"feruzaspeech","name":"FeruzaSpeech","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","34 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on NavAI: 34 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fev-bench","name":"fev-bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (35 entries)."],["scores.models","pass","35 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 35 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fever","name":"FEVER","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fgvc-aircraft","name":"FGVC-Aircraft","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (58 entries)."],["scores.models","pass","1 model we track, plus 57 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 39.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MTEB: 1 row for models we track and 57 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fiction-bench-foreverse","name":"Fiction Bench (Foreverse)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Terra."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Terra (#18)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on foreverse.app: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fiction-bench-story-hacker","name":"Fiction Bench (Story Hacker)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","36 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on bench.storyhacker.ai: 36 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fiction-livebench","name":"Fiction.LiveBench","summary":{"pass":3,"warn":3,"fail":0,"info":4,"text":"3 of 6 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from boards Epoch AI copied (57 models)."],["scores.models","pass","57 models we track."],["scores.recent","info","The newest came 10 months ago, in January 2026, for Kimi K2.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 97.2%, past 85% since April 2025."],["import.importer","warn","The record links a board on fiction.live that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fieldworkarena-2","name":"FieldWorkArena V2.0","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 52%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"filbench","name":"FilBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bio-adversarial-production-prompts-openai","name":"Filtered, adversarial sample of production prompts","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5 mini. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 96.9%, past 85% since July 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fin-eva","name":"Fin-Eva","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fin-fact","name":"Fin-Fact","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","3 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Claude 3 Opus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 61%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-finance-bench-moonshot","name":"Finance Bench (Moonshot)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 62.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finance-fundamentals-kensho","name":"Finance Fundamentals","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on benchmarks.kensho.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finance-modeling","name":"Finance Modeling","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for MiniMax-M2.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 21.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finance-modeling-pro-minimax","name":"Finance Modeling Pro","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for MiniMax-M2.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 57%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkel-finance-reasoning","name":"Finance Reasoning (Snorkel)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","21 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 53.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snorkel.ai: 21 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"financebench","name":"FinanceBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Llama 2-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-70B (#210)."],["scores.headroom","pass","The best is 41%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"financereasoning","name":"FinanceReasoning","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries) and system cards (1 score)."],["scores.models","pass","10 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 13 months ago, in October 2025, for Ring-1T. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1 (#118)."],["scores.headroom","info","Saturated. The best is 87.4%, past 85% since September 2024."],["import.importer","pass","We read the board on bupt-reasoning-lab.github.io: 17 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"financial-coaching-bench","name":"Financial Coaching Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 76.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 28 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"financial-phrasebank","name":"Financial PhraseBank","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (193 entries)."],["scores.models","pass","5 models we track, plus 188 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 95.2%, past 85% since June 2025."],["import.importer","pass","We read the board on MTEB: 5 rows for models we track and 188 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"daloopa-financial-retrieval","name":"Financial Retrieval (Daloopa)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 91.6%, past 85% since August 2025."],["import.importer","pass","We read the board on daloopa.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"financial-statement-ocr-ramp","name":"Financial Statement OCR","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on builders.ramp.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fs-review","name":"Financial Statement Review (fs-review)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finben","name":"FinBen","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","6 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Llama 3-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-8B (#206)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finch-finworkbench","name":"Finch (FinWorkBench)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries) and launch posts (1 score)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from October 2026, for Mistral Large 4 (Le Chonk, open weights)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 73.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fincrime-agent-benchmark-castellum","name":"FinCrime Agent Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","9 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 71.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on castellum.ai: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"find-kensho","name":"FIND","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"findeepindicator","name":"FinDeepIndicator","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Grok 4.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 4.6 (#57)."],["scores.headroom","pass","The best is 39%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openfinarena.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"findendro","name":"FinDendro","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-9B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"findesk-bench","name":"FinDesk-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","5 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Saturated. The best is 90.4%, past 85% since October 2026."],["import.importer","pass","We read the board on veris.ai: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"findver","name":"FinDVer","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finesse-bench","name":"FINESSE-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries)."],["scores.models","pass","36 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on limexailab.github.io: 36 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fineval","name":"FinEval","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","7 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Qwen2.5-72B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 69.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 7 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fineval-6","name":"FinEval 6.0","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-30B-A3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 84.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 5 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fineva-reranking","name":"FinEvaReranking","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finfact-reranking","name":"FinFactReranking","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finfirst","name":"FinFIRST","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from September 2026, for Ling-3.0-flash-Fin."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 71.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fingreat","name":"FinGREAT","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (44 entries)."],["scores.models","pass","35 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 92.8%, past 85% since February 2026."],["import.importer","pass","We read the board on arxiv.org: 35 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finhopper","name":"FinHopper","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","9 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GLM-4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 67.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 32 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finmaster","name":"FinMaster","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for GPT-4.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 73%, under 85%, so it still separates models."],["import.importer","pass","We read the board on jianginsight.github.io: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finmem-trading","name":"FinMem Trading","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 4 years ago, in November 2022, for davinci-003."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 1.63%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finmteb","name":"FinMTEB","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finmteb-final","name":"FinMTEB FINAL","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finnish-parliament-asr","name":"Finnish Parliament ASR test","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","4 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for NVIDIA Nemotron 3.5 Transcribe 30B-A4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finqa","name":"FinQA","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on codalab.lisn.upsaclay.fr: 4 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finqa-snorkel-openenv-environment","name":"FinQA (Snorkel OpenEnv environment)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","22 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 81%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snorkel.ai: 22 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finred-financial-safety","name":"FinRED Financial Safety","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","5 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 24 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finsbd-2021","name":"FinSBD 2021","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on competitions.codalab.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finsearchcomp","name":"FinSearchComp","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries), launch posts (7 scores) and system cards (3 scores)."],["scores.models","pass","12 models we track, plus 13 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for LongCat-2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K2.5 (#76)."],["scores.headroom","pass","The best is 81.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 11 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finsearchcomp-global","name":"FinSearchComp Global","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for MiniMax-M2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 65.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on randomtutu.github.io: 12 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finsearchcomp-greater-china","name":"FinSearchComp Greater China","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on randomtutu.github.io: 12 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finstepbench","name":"FinStepBench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for Step 5 Preview."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 63.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finstepbench-corporatevaluation","name":"FinStepBench CorporateValuation","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for Step 5 Preview."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 77.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on stepfun.com: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finstepbench-deepresearch","name":"FinStepBench DeepResearch","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for Step 5 Preview."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 59.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on stepfun.com: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finstepbench-livesearch","name":"FinStepBench LiveSearch","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for Step 5 Preview."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 76.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on stepfun.com: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finsts","name":"FinSTS","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fintoolbench","name":"FinToolBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fiqa-sa-bloomberggpt","name":"FiQA SA (BloombergGPT protocol)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","1 model we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in June 2023, for GPT-4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fiqa2018-reranking","name":"FiQA2018Reranking","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fire360","name":"Fire360","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 53.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on fire360bench.github.io: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"firecrawl-search-bench","name":"Firecrawl Search Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 79%, under 85%, so it still separates models."],["import.importer","pass","We read the board on firecrawl.dev: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"firefox-147-exploit-evaluation-anthropic","name":"Firefox 147 exploit evaluation (Anthropic)","summary":{"pass":5,"warn":1,"fail":0,"info":4,"text":"5 of 6 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 72.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fireworks-specialized-intelligence-index","name":"Fireworks Specialized Intelligence Index","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","warn","The record links a board on fireworks.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"first-proof","name":"First Proof","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"first-proof-batch-2","name":"First Proof Second Batch","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for GPT-5.5 Pro. That’s expected for a test replaced by First Proof Third Batch."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 Pro (#15)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on 1stproof.org: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"first-proof-batch-3","name":"First Proof Third Batch","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","2 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on 1stproof.org: 3 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"first-person-fairness-evaluation","name":"First-Person Fairness evaluation","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (13 scores)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"five-bench-video-editing","name":"FiVE-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flageval-arena","name":"FlagEval Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","28 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on flageval.baai.ac.cn: 33 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flageval-debate","name":"FlagEval Debate","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","10 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for Yi-Lightning."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on flageval.baai.ac.cn: 10 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flame-react-eval","name":"Flame-React-Eval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries), launch posts (4 scores) and system cards (3 scores)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for GLM-5V-Turbo. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o (#167)."],["scores.headroom","info","Saturated. The best is 93.8%, past 85% since December 2025."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flames-chinese","name":"FLAMES","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","5 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in October 2023, for ChatGLM3-6B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen-14B (#214)."],["scores.headroom","pass","The best is 43.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on flames.opencompass.org.cn: 5 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flash-fiction-stylistic-diversity-lech-mazur","name":"Flash Fiction Stylistic Diversity","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","23 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 24 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flashdexretarget-50-motion-benchmark","name":"FlashDexRetarget 50-motion benchmark","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on holiday-robot.github.io: 4 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flashinfer-bench","name":"FlashInfer-Bench","summary":{"pass":6,"warn":2,"fail":0,"info":4,"text":"6 of 8 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","pass","We read the board on bench.flashinfer.ai: 3 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flavourbench-complete-core","name":"FlavourBench Complete Core","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","24 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 24 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flavourbench-powered-v43","name":"FlavourBench Powered v43","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","17 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for DeepSeek V4 Pro 0813."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 17 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flawed-fictions","name":"Flawed Fictions","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","10 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 17 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flenqa","name":"FlenQA","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Phi-4-Reasoning-plus. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 97.9%, past 85% since April 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fleurs","name":"FLEURS","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (12 scores) and system cards (9 scores)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flex-rsi","name":"Flex-RSI","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on flex-rsi.github.io: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flickr30k","name":"Flickr30k","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-floorplan-arena","name":"Floorplan Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","8 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 11 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flores","name":"Flores","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (4 scores) and system cards (5 scores)."],["scores.models","pass","8 models we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Aya Vision 32B. That’s expected for a test replaced by FLORES-200."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flores-plus","name":"FLORES+","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Glimmer."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flores-200","name":"FLORES-200","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Hunyuan-MT-7B. That’s expected for a test replaced by FLORES+."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flow-bench-ibm","name":"FLOW-BENCH","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"flteval","name":"FLTEval","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries), launch posts (4 scores) and system cards (2 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for Leanstral 1.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mistral.ai: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fml-bench","name":"FML-bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"foambench","name":"FoamBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash. That’s expected for a test replaced by FoamBench V2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.5 Flash (#112)."],["scores.headroom","pass","The best is 22.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on nithinsomu.github.io: 3 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"foambench-2","name":"FoamBench V2","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 86.5%, past 85% since February 2026."],["import.importer","pass","We read the board on nithinsomu.github.io: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"focusbench-meta","name":"FocusBench (Meta)","summary":{"pass":11,"warn":1,"fail":0,"info":0,"text":"11 of 12 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and launch posts (2 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 54.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on research.meta.ai: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"followbench","name":"FollowBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","3 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for GPT-3.5 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","pass","The best is 73.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"followir","name":"FollowIR","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (212 entries)."],["scores.models","pass","4 models we track, plus 208 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MTEB: 4 rows for models we track and 208 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fone","name":"FoNE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"food-101","name":"Food-101","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (49 entries)."],["scores.models","pass","49 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on MTEB: 49 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"foodtruck-bench","name":"FoodTruck Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (66 entries)."],["scores.models","pass","65 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","3 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on foodtruckbench.com: 65 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"foqa","name":"FoQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (336 entries)."],["scores.models","pass","91 models we track, plus 245 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Nemotron 3 Nano 4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-235B-A22B (#131)."],["scores.headroom","pass","The best is 79.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on EuroEval: 103 rows for models we track and 245 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"forb","name":"FORB","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"forecastbench","name":"ForecastBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (429 entries) and launch posts (2 scores)."],["scores.models","pass","86 models we track, plus 343 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a Brier score, with no fixed ceiling."],["import.importer","pass","We read the board on forecastbench.org: 183 rows for models we track and 343 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"forensics-ctf-anthropic","name":"Forensics CTF challenges (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since November 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"formal-conjectures","name":"Formal Conjectures","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"formal-logic-deduction","name":"Formal Logic Deduction","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"formal-imo","name":"Formal-IMO","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"formally-verified-c","name":"Formally Verified C","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"formalmath","name":"FormalMATH","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries) and system cards (1 score)."],["scores.models","pass","2 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for DeepSeek-Prover-V2-671B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 28.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sphere-ai-lab.github.io: 2 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"formalmath-lite","name":"FormalMATH Lite","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","2 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for DeepSeek-Prover-V2-671B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 61.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sphere-ai-lab.github.io: 3 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"formextract-bench","name":"FormExtract Bench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on kaggle.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"forte","name":"FORTE","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries) and launch posts (2 scores)."],["scores.models","pass","22 models we track."],["scores.recent","pass","The newest is from November 2026, for LongCat-2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 83.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on agi-eval-official.github.io: 21 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"forte-mm","name":"FORTE-MM","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 74.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fortisavqa","name":"FortisAVQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","1 model we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for VITA."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 48.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 12 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fortress","name":"FORTRESS","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (82 entries), launch posts (1 score) and system cards (3 scores)."],["scores.models","pass","82 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on labs.scale.com: 89 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fortress-rn","name":"FORTRESS-RN","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","info","Saturated. The best is 97.9%, past 85% since July 2026."],["import.importer","warn","The record links a board on labs.scale.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fosent","name":"FoSent","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (540 entries)."],["scores.models","pass","147 models we track, plus 393 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 91.6%, past 85% since June 2025."],["import.importer","pass","We read the board on euroeval.com: 165 rows for models we track and 393 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"founder-bench-eico","name":"Founder Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eico.so: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fox-document-benchmark","name":"Fox Document Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fpl-2026-agentic-fpga-backend-optimization","name":"FPL 2026 Agentic FPGA Backend Optimization Competition","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","24 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on xilinx.github.io: 24 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fquad","name":"FQuAD","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fquad-1-1","name":"FQuAD 1.1","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fquad-2","name":"FQuAD 2.0","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frames","name":"FRAMES","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry), launch posts (6 scores) and system cards (6 scores)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Seed2.0 Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K2 Thinking (#94)."],["scores.headroom","info","Saturated. The best is 87.6%, past 85% since November 2025."],["import.importer","pass","We read the board on huggingface.co: 1 entry at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fraudbench-goal-lab","name":"FraudBench (GOAL Lab)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries)."],["scores.models","pass","6 models we track, plus 14 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Grok 4.1 Fast."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#112)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"freihand","name":"FreiHAND","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","32 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on opencodepapers-b7572d.gitlab.io: 32 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fijo-insurance-soft-skills","name":"French Insurance Job Offer","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frenchbench-croissantllm","name":"FrenchBench (CroissantLLM)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"freshqa","name":"FreshQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-4 Turbo (Jan 2024)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 80.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"freshstack","name":"FreshStack","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (62 entries)."],["scores.models","pass","2 models we track, plus 60 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on fresh-stack.github.io: 2 rows for models we track and 60 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fretbench","name":"FretBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Opus 4.7. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since December 2025."],["import.importer","pass","We read the board on fretbench.tymo.ai: 21 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frogsgame-post-training-task","name":"FrogsGame post-training task","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 34%, under 85%, so it still separates models."],["import.importer","pass","We read the board on frontierswe.com: 17 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontier-agentic-ml-ds-turing","name":"Frontier Agentic ML & DS","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 64.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.turing.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontier-agentic-stem-turing","name":"Frontier Agentic STEM","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 55.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.turing.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontier-design-benchmark","name":"Frontier Design Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","15 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on contralabs.com: 15 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontier3dbench","name":"Frontier3DBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from November 2026, for Doubao Seed Evolving."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on lab.einsia.ai: 11 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontierbioml","name":"FrontierBioML","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","5 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on frontierbioml.com: 14 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontierbreach","name":"FrontierBreach","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","22 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on PromptArmor: 22 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiercad","name":"FrontierCAD","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on normal.ai: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiercad-assemblies","name":"FrontierCAD Assemblies","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on normal.ai: 7 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiercad-parts","name":"FrontierCAD Parts","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on normal.ai: 7 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontierchallenge","name":"FrontierChallenge","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for DeepSeek V4 Pro 0813."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 20.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on apodexai.github.io: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiercode","name":"FrontierCode","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (12 entries), launch posts (4 scores) and system cards (2 scores)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 54.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cognition.com: 31 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiercode-1-1","name":"FrontierCode 1.1","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries), launch posts (10 scores) and system cards (7 scores)."],["scores.models","pass","21 models we track."],["scores.recent","pass","The newest is from November 2026, for SWE-2 Flash."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 64.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cognition.com: 36 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiercode-2-0","name":"FrontierCode 2.0","summary":{"pass":11,"warn":1,"fail":0,"info":0,"text":"11 of 12 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (35 entries), launch posts (10 scores) and system cards (4 scores)."],["scores.models","pass","29 models we track, plus 11 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","8 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 50.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cognition.com: 139 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 5 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiercs","name":"FrontierCS","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on frontier-cs.org: 15 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiercs-1-1","name":"FrontierCS 1.1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on frontier-cs.org: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiercs-2","name":"FrontierCS 2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","20 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Qwen4-Max."],["scores.frontier","pass","8 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 58.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 20 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiercyber-irregular","name":"FrontierCyber","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (5 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 38.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontier-ee-bench","name":"FrontierEE Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","5 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 36.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on evidentmachines.com: 5 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontierfinance","name":"FrontierFinance","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries) and launch posts (2 scores)."],["scores.models","pass","18 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 66.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on samaya.ai: 20 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiermath","name":"FrontierMath","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (69 entries), Epoch AI’s runs (70 models), launch posts (12 scores) and system cards (1 score)."],["scores.models","pass","75 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for MAI-Code-1-Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 Pro (#15)."],["scores.headroom","pass","The best is 52.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on epoch.ai: 95 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 10 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiermath-erdos","name":"FrontierMath Erdős","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries), Epoch AI’s runs (16 models) and system cards (1 score)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","8 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 10.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on epoch.ai: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiermath-tier-4","name":"FrontierMath Tier 4","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (54 entries), Epoch AI’s runs (55 models) and launch posts (6 scores)."],["scores.models","pass","59 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 97.6%, past 85% since September 2026."],["import.importer","pass","We read the board on epoch.ai: 70 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 5 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiermath-tier-4-v2","name":"FrontierMath Tier 4 v2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (77 entries), Epoch AI’s runs (76 models) and launch posts (3 scores)."],["scores.models","pass","78 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since June 2026."],["import.importer","pass","We read the board on epoch.ai: 81 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiermath-tiers-1-3-v2","name":"FrontierMath Tiers 1-3 v2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (94 entries), Epoch AI’s runs (94 models) and launch posts (3 scores)."],["scores.models","pass","96 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 95.1%, past 85% since April 2026."],["import.importer","pass","We read the board on epoch.ai: 126 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiermath-open-problems","name":"FrontierMath: Open Problems","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on epoch.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontiermemory","name":"FrontierMemory","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 73.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on frontiermemory.com: 76 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontier-ml","name":"FrontierML","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on frontierml.org: 57 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontierml-synthetic-sciences","name":"FrontierML (Synthetic Sciences)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on frontierml.ai: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontierphysics","name":"FrontierPhysics","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","22 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 27.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchflow.ai: 26 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontier-rsi","name":"FrontierRSI","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","6 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 34.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on frontier-rsi.org: 9 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontierscience","name":"FrontierScience","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (8 scores) and system cards (1 score)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from November 2026, for LongCat-2.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 87%, past 85% since September 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontierscience-olympiad","name":"FrontierScience-Olympiad","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries) and system cards (2 scores)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 77.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openai.com: 12 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontierscience-research","name":"FrontierScience-Research","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries), launch posts (3 scores) and system cards (2 scores)."],["scores.models","pass","12 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Pro (#19)."],["scores.headroom","pass","The best is 38%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openai.com: 12 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontierswe","name":"FrontierSWE","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries), launch posts (4 scores) and system cards (9 scores)."],["scores.models","pass","28 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 75.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on frontierswe.com: 26 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 8 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"frontierswe-v1","name":"FrontierSWE v1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries), launch posts (4 scores) and system cards (2 scores)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 88.2%, past 85% since June 2026."],["import.importer","pass","We read the board on frontierswe.com: 17 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fruit-box","name":"Fruit Box","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fsd50k","name":"FSD50K","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on Scholar Compass: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fsdkaggle2018","name":"FSDKaggle2018","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries)."],["scores.models","pass","30 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on dcase.community: 30 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fsdkaggle2019","name":"FSDKaggle2019","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on dcase.community: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"full-duplex-bench","name":"Full-Duplex-Bench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Grok Voice 3.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"full-duplex-bench-1-5","name":"Full-Duplex-Bench v1.5","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for GPT-Live-1."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"full-duplex-bench-2","name":"Full-Duplex-Bench v2","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for gpt-realtime. That’s expected for a test replaced by Full-Duplex-Bench v3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ericsunkuan.github.io: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"full-duplex-bench-3","name":"Full-Duplex-Bench v3","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for TML-Interaction-Small."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 68%, under 85%, so it still separates models."],["import.importer","pass","We read the board on daniellin94144.github.io: 6 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fullstackbench","name":"FullStackBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (28 entries), launch posts (6 scores) and system cards (5 scores)."],["scores.models","pass","18 models we track, plus 18 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3-Coder-Next."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5-35B-A3B (#103)."],["scores.headroom","pass","The best is 68.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 10 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"functional-math","name":"Functional MATH","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries) and launch posts (1 score)."],["scores.models","pass","8 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for Gemini 1.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#174)."],["scores.headroom","pass","The best is 64.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 49 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fund-smart-routing-agent-ramp","name":"Fund Smart Routing Agent","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on builders.ramp.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fundus-mmbench","name":"Fundus-MMBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fungraph3d","name":"FunGraph3D","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"furniture-assembly-benchmark","name":"Furniture Assembly Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries) and Epoch AI’s runs (41 models)."],["scores.models","pass","41 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 90%, past 85% since November 2026."],["import.importer","pass","We read the board on epoch.ai: 39 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"futureeval-metaculus","name":"FutureEval (Metaculus)","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on metaculus.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"futureomni","name":"FutureOmni","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries) and launch posts (1 score)."],["scores.models","pass","4 models we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for MiMo-V2-Omni."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 66.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openmoss.github.io: 3 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"futurex","name":"FutureX","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","7 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Grok 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4 (#86)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on futurex.live: 11 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fveval","name":"FVEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-70B (#177)."],["scores.headroom","pass","The best is 24.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"g2-voice-ai-agent-platform-evaluations","name":"G2 Voice AI Agent Platform Evaluations","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ai.g2.com: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gaia","name":"GAIA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3,699 entries), launch posts (5 scores) and system cards (3 scores)."],["scores.models","pass","7 models we track, plus 3,697 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Intern-S1-Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 87.4%, past 85% since December 2025."],["import.importer","pass","We read the board on huggingface.co: 440 rows for models we track and 3,263 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gaia2-cli","name":"Gaia2-CLI","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 57%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 7 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gaik-ajokortti","name":"GAIK Ajokortti transcription test","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","9 models we track, plus 16 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-Transcribe."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 10 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gaik-la-murre","name":"GAIK LA-murre dialect test","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","4 models we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.5 Flash (#44)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gaik-skn-dialects","name":"GAIK SKN dialect test","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","4 models we track, plus 16 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.5 Flash (#44)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"galahad-vla","name":"Galahad","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on xn--7xa.monster: 2 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"galcola","name":"GalCoLA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"galileo-agent-leaderboard","name":"Galileo Agent Leaderboard","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","29 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for o3. That’s expected for a test replaced by Galileo Agent Leaderboard v2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 29 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"galileo-agent-leaderboard-v2","name":"Galileo Agent Leaderboard v2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","21 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for Mistral Medium 3.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4 (#86)."],["scores.headroom","pass","The best is 62%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 21 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"galileo-hallucination-index","name":"Galileo LLM Hallucination Index","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","9 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in November 2023, for GPT-3.5 Turbo. That’s expected for a test replaced by Galileo LLM Hallucination Index RAG Special."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"galileo-hallucination-index-rag","name":"Galileo LLM Hallucination Index RAG Special","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for Claude 3.5 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"game-arena-three-check-chess-generality","name":"Game Arena 3 Check Chess (Generality)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Kimi K2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on game-arena.ai: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"game-arena-anti-chess-generality","name":"Game Arena Anti Chess (Generality)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Kimi K2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on game-arena.ai: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"game-arena-atomic-chess-generality","name":"Game Arena Atomic Chess (Generality)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","9 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on game-arena.ai: 11 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-bargaining","name":"Game Arena Bargaining","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","39 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 39 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-checkers","name":"Game Arena Checkers","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries)."],["scores.models","pass","41 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 41 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-chess","name":"Game Arena Chess","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","42 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 42 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-chess-openings","name":"Game Arena Chess Openings","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 11 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-clobber","name":"Game Arena Clobber","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","39 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 39 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-coin-game","name":"Game Arena Coin Game","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","39 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 39 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"game-arena-crazyhouse-chess-generality","name":"Game Arena Crazyhouse Chess (Generality)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Kimi K2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on game-arena.ai: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-dark-hex","name":"Game Arena Dark Hex","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries)."],["scores.models","pass","41 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 41 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-dots-and-boxes","name":"Game Arena Dots and Boxes","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries)."],["scores.models","pass","40 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 40 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-five-in-a-row","name":"Game Arena Five in a Row","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 20 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-four-in-a-row","name":"Game Arena Four in a Row","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (43 entries)."],["scores.models","pass","43 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 43 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"game-arena-four-in-a-row-generality","name":"Game Arena Four-in-a-Row (Generality)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","22 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Kimi K2 Thinking."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on game-arena.ai: 52 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-go","name":"Game Arena Go","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","39 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 39 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"game-arena-horde-chess-generality","name":"Game Arena Horde Chess (Generality)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Kimi K2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on game-arena.ai: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"game-arena-king-of-the-hill-generality","name":"Game Arena King of the Hill (Generality)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Kimi K2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on game-arena.ai: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-lines-of-action","name":"Game Arena Lines of Action","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","39 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 39 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-nine-mens-morris","name":"Game Arena Nine Men's Morris","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","39 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 39 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-poker","name":"Game Arena Poker","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","31 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 31 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"game-arena-racing-kings-generality","name":"Game Arena Racing Kings (Generality)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Kimi K2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on game-arena.ai: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-reversi","name":"Game Arena Reversi","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","39 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 39 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-ultimate-tic-tac-toe","name":"Game Arena Ultimate Tic-Tac-Toe","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","39 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 39 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-werewolf","name":"Game Arena Werewolf","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-word-art","name":"Game Arena Word Art","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","39 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 39 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena-word-association","name":"Game Arena Word Association","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries)."],["scores.models","pass","40 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 40 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gameadsbench","name":"GameAdsBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on video-arena.com: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gameasg-bench","name":"GameASG-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 55.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 16 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gamecraft-bench","name":"GameCraft-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","9 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Sol."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 75.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 15 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gamedevbench","name":"GameDevBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 70.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on waynechi.com: 21 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gameenginebench","name":"GameEngineBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 55.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on nitrode.com: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gameforgebench","name":"GameForgeBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gamehorizon","name":"GameHorizon","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (44 entries)."],["scores.models","pass","38 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on gamehorizon-suite.github.io: 38 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gameworld","name":"GameWorld","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries) and system cards (1 score)."],["scores.models","pass","24 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6.1 Terra (#7)."],["scores.headroom","pass","The best is 76.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on gameworld-project.github.io: 38 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gamexpert-bench","name":"GameXpert-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kwen-chen.github.io: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gaokao-math-2026","name":"Gaokao Math 2026","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Spark-X2.6-4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gaokao-bench","name":"GAOKAO-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","10 models we track, plus 5 other entries on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for InternVL3.5-241B-A28B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Saturated. The best is 94.5%, past 85% since August 2025."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gaokao-eval-2024","name":"GAOKAO-Eval 2024","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","3 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for Qwen2-72B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gap-graph-as-policy","name":"GaP: Graph-as-Policy","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"garage","name":"GaRAGe","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","9 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (October 2024) (#150)."],["scores.headroom","pass","The best is 60.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"garble-bench-voicearena","name":"Garble-Bench (Voice Arena)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 8 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"garblebench","name":"GarbleBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 55.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on garble-bench.adamholter.chatgpt.site: 29 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gatemem","name":"GateMem","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on rzhub.github.io: 35 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gauntlet","name":"Gauntlet","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gauss-math","name":"GAUSS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gbaeval","name":"GBAEval","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","24 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 90.6%, past 85% since September 2026."],["import.importer","pass","We read the board on gbaeval.com: 24 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gbench","name":"gBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gd-vcr","name":"GD-VCR","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdm-situational-awareness","name":"GDM Situational Awareness","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (8 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 90.9%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdm-stealth","name":"GDM Stealth","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdp-pdf","name":"GDP.pdf","summary":{"pass":11,"warn":1,"fail":0,"info":0,"text":"11 of 12 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries), launch posts (15 scores) and system cards (10 scores)."],["scores.models","pass","31 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","7 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 83.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on surgehq.ai: 21 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 7 pairs differs by more than 10 points, the largest for Muse Wonder (−24 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdp-xlsx","name":"GDP.xlsx","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (35 entries)."],["scores.models","pass","34 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 43.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on surgehq.ai: 34 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdpevo","name":"GDPevo","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 67.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 16 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdpval","name":"GDPval","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries), launch posts (10 scores) and system cards (2 scores)."],["scores.models","pass","15 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 Pro (#15)."],["scores.headroom","info","Saturated. The best is 87.9%, past 85% since June 2026."],["import.importer","pass","We read the board on openai.com: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdpval-extended-mercor","name":"GDPval Extended","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (44 entries)."],["scores.models","pass","43 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 7.10%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mercor.com: 44 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdpval-v2-openai","name":"GDPval v2 (OpenAI)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 50.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdpval-aa","name":"GDPval-AA","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (15 scores) and system cards (14 scores)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdpval-aa-v2","name":"GDPval-AA v2","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (26 scores) and system cards (17 scores)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Glimmer."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdpval-aa-v2-1","name":"GDPval-AA v2.1","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (11 scores) and system cards (6 scores)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdpval-diamond-bytedance","name":"GDPVal-Diamond","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Seed2.0 Pro. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 21.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdpval-mm","name":"GDPval-MM","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (1 score)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for MiniMax-M2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 73.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdpval-rubrics-minimax","name":"GDPval-Rubrics (MiniMax)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is MiniMax-M3 (#80)."],["scores.headroom","pass","The best is 74.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdpworks","name":"GDPWorks","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdpx","name":"GDPx","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","16 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 16.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on gdpx.semianalysis.com: 20 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gebench-stepfun","name":"GEBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on stepfun-ai.github.io: 16 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gecko","name":"Gecko","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geckonum","name":"GeckoNum","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","1 model we track, plus 11 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 11 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gedit-bench","name":"GEdit-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Step1X-Edit v1.2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on raw.githubusercontent.com: 5 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gemini-agentic-alignment-evals","name":"Gemini agentic alignment evals","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gemini-content-safety-evals","name":"Gemini content safety evals","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gemini-personality-evaluation-sycophancy","name":"Gemini Personality Evaluation (sycophancy)","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gemini-prompt-injection-evaluation-suites","name":"Gemini prompt-injection evaluation suites","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gemini-robotics-generalization-google","name":"Gemini Robotics generalization benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robotics-on-device-data-scaling-google","name":"Gemini Robotics On-Device data scaling evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gen-bench-dynamic-scheduling","name":"GEN-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","5 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3.2 (#91)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"genai-arena","name":"GenAI-Arena","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","1 model we track, plus 41 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 41 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"genai-arena-image-edit","name":"GenAI-Arena Image Editing","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 10 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"genai-arena-image-generation","name":"GenAI-Arena Image Generation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","1 model we track, plus 16 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 16 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"genai-arena-video-generation","name":"GenAI-Arena Video Generation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 15 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"genai-bench","name":"GenAI-Bench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Flash Image (nano-banana-2, Web Search)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt24-speaker-listener-gender-resolution","name":"Gender Resolution in Speaker-Listener Dialogue Roles","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gender1person","name":"GENDER1PERSON","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"genebench","name":"GeneBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 Pro (#15)."],["scores.headroom","pass","The best is 33.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"genebench-pro","name":"GeneBench Pro","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries), launch posts (4 scores) and system cards (1 score)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-Rosalind-6."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 37.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openai.com: 45 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 3 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"general-agent-benchmark-moonshot","name":"General Agent Benchmark (Moonshot)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"general-assistant-inhouse-minimax","name":"General Assistant (MiniMax in-house)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for MiniMax-Text-01. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 73.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"general-refusals-xai","name":"General refusals (xAI)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (8 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"general-bench","name":"General-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (101 entries)."],["scores.models","pass","41 models we track, plus 60 other entries on its leaderboard."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for Falcon3-7B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on generalist.top: 42 rows for models we track and 60 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"general365","name":"General365","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","23 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5 397B-A17B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 62.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on general365.github.io: 25 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cdt-voting-with-a-disability","name":"Generating Confusion: Voting with a Disability","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","warn","The record links a board on cdt.org that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geneval","name":"GenEval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","3 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for ERNIE 5.0. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 90.1%, past 85% since May 2025."],["import.importer","pass","We read the board on github.com: 6 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geneval-2","name":"GenEval 2","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"genexam","name":"GenExam","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"genomegym-brca1","name":"GenomeGym-BRCA1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"genomeqa","name":"GenomeQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 63.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"genpuzzle","name":"GenPuzzle","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Flash Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"genscale","name":"GenScale","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"genspark-slides-benchmark","name":"Genspark Slides Benchmark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 83.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on fireworks.ai: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geo-bench-earth-monitoring-2","name":"GEO-Bench-2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 14 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geobench","name":"GeoBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (53 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","55 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 95%, past 85% since May 2025."],["import.importer","pass","We read the board on geobench.org: 59 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geobench-vlm","name":"GEOBench-VLM","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","1 model we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on the-ai-alliance.github.io: 1 row for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geobenchx","name":"GeoBenchX","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","info","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet. That’s expected for a test replaced by GeoBenchX v2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 51.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geobenchx-2","name":"GeoBenchX v2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (50 entries)."],["scores.models","pass","49 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 84%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Axis Spatial: 95 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geobiaset","name":"GeoBiaset","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","5 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Qwen3-VL-8B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","pass","The best is 69.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on miccunifi.github.io: 5 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geode","name":"GeoDE","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geodrive-bench","name":"GeoDrive-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 91.9%, past 85% since September 2025."],["import.importer","pass","We read the board on github.com: 27 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geoexpertqueries-google","name":"GeoExpertQueries","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geoflip","name":"GeoFlip","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","2 models we track, plus 14 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Wan2.2-I2V-A14B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geoguessbench-nutlope","name":"GeoGuessBench (Nutlope)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on geoguessbench.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geoguessbench-panoramic","name":"GeoGuessBench (panoramic)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on geo-llm-three.vercel.app: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geoguesser-bench-berkeley","name":"GeoGuesser Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","23 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huanzhimao.com: 23 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geoguesser-environment-fineenvs","name":"GeoGuesser environment (FineEnvs)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 5 (#35)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 15 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geoktools-google","name":"GeoKTools","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geomap-bench","name":"GeoMap-Bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on microsoft.github.io: 1 row for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geomind","name":"GeoMind","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","3 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for π0.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 20.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geomlama","name":"GeoMLAMA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geonatureagent","name":"GeoNatureAgent Benchmark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for GLM-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GLM-5 (#89)."],["scores.headroom","pass","The best is 60.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"geosound","name":"GeoSound","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gerdalir","name":"GerDaLIR","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gerdalir-small","name":"GerDaLIR Small","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries)."],["scores.models","pass","4 models we track, plus 33 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on VAGO Solutions: 4 rows for models we track and 33 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gerlayqa","name":"GerLayQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","4 models we track, plus 29 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on VAGO Solutions: 4 rows for models we track and 29 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gerlerb","name":"GerLeRB","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","4 models we track, plus 29 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 4 rows for models we track and 29 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"german-argument-mining","name":"German Argument Mining","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dri-german-federal-election-chatbot-audit-2025","name":"German federal election chatbot audit","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"german-fruit-story-antipresuppositions","name":"German fruit-story antipresuppositions","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","9 models we track, plus 5 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since April 2025."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"german-traffic-sign-recognition-benchmark","name":"German Traffic Sign Recognition Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"germanquad","name":"GermanQuAD","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (271 entries)."],["scores.models","pass","88 models we track, plus 183 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Nemotron 3 Nano 4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-235B-A22B (#131)."],["scores.headroom","pass","The best is 70%, under 85%, so it still separates models."],["import.importer","pass","We read the board on EuroEval: 101 rows for models we track and 183 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"germeval-2014-named-entity-recognition","name":"GermEval 2014 Named Entity Recognition","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gg-bench","name":"gg-bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 36.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ghostink-honest-ocr-on-bill-photos","name":"GhostInk: Honest OCR on Bill Photos","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Grok 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4.5 (#47)."],["scores.headroom","pass","The best is 0%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gie-bench","name":"GIE-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gift-eval","name":"GIFT-Eval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (173 entries)."],["scores.models","pass","173 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 173 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gift-eval-ctx","name":"GIFT-Eval-CTX","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","2 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 5 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"in-the-wild-asr-gigaam-sber","name":"GigaAM Multilingual Internal In-the-Wild Speech Tests","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gigaspeech","name":"GigaSpeech","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gim","name":"GIM","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (21 entries)."],["scores.models","pass","16 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Claude Opus 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Pro (#19)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 38 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gittaskbench","name":"GitTaskBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-14B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 48.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 30 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gjcool-vertical-book-ocr","name":"GJ.cool Vertical Book OCR Evaluation","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","2 models we track, plus 5 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Qwen3.5-4B. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 89%, past 85% since January 2026."],["import.importer","pass","We read the board on ocr.gj.cool: 2 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"glee","name":"GLEE","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (489 entries)."],["scores.models","pass","9 models we track, plus 480 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on glee-competition.com: 13 rows for models we track and 476 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gles-2017-vote-choice","name":"GLES 2017 vote choice","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"glianorex","name":"Glianorex","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","12 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for DeepSeek-V3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3 (#141)."],["scores.headroom","pass","The best is 71.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 15 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"all-tools-information-seeking-zai","name":"GLM-4 All Tools information seeking (Z.ai)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries) and system cards (1 score)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for GLM-4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","pass","The best is 78.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"global-mmlu","name":"Global MMLU","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (13 scores)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 96%, past 85% since February 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"global-mmlu-lite","name":"Global MMLU Lite","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (44 entries), launch posts (12 scores) and system cards (12 scores)."],["scores.models","pass","56 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 95.7%, past 85% since February 2025."],["import.importer","pass","We read the board on Kaggle: 45 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 4 pairs differs by more than 10 points, the largest for Gemma 3 4B (−11 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"global-mmlu-lite-2","name":"Global MMLU Lite 2.0","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 11 months ago, in December 2025, for Nemotron 3 Nano 30B-A3B Base. That’s expected for a test replaced by Global MMLU Lite 3.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 74.5%, under 85%, so it still separates models."],["import.importer","warn","The record links a board on kaggle.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"global-mmlu-lite-3","name":"Global MMLU Lite 3.0","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (44 entries), launch posts (3 scores) and system cards (4 scores)."],["scores.models","pass","46 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 95.7%, past 85% since April 2025."],["import.importer","pass","We read the board on kaggle.com: 43 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"global-piqa","name":"Global PIQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (177 entries), launch posts (14 scores) and system cards (1 score)."],["scores.models","pass","68 models we track, plus 118 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 93.9%, past 85% since November 2025."],["import.importer","pass","We read the board on github.com: 59 rows for models we track and 118 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 5 pairs differs by more than 10 points, the largest for Qwen3.5-4B (+17 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"global-mgsm","name":"GlobalMGSM","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"globebench","name":"GlobeBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","19 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 49.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 19 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"glue","name":"GLUE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 11 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"glyphbench","name":"GlyphBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 27B (#59)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gmai-mmbench","name":"GMAI-MMBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","13 models we track, plus 16 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for InternVL-Chat-V1-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 43.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 13 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gmat-quantitative","name":"GMAT Quantitative","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries) and system cards (3 scores)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in November 2024, for Llama 3.1-405B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Saturated. The best is 96%, past 85% since July 2024."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gmat-verbal","name":"GMAT Verbal","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 2 years ago, in November 2024, for Llama 3.1-405B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Saturated. The best is 86.6%, past 85% since November 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gold-commodity-news","name":"Gold Commodity News","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"golden3-microsoft","name":"Golden3 speech summarization (Microsoft)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 20 months ago, in March 2025, for Phi-4-multimodal. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"goliath-human-mesh-recovery","name":"Goliath human mesh recovery","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gomokubench-homer-quan","name":"GomokuBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","9 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"goodai-ltm-benchmark","name":"GoodAI LTM Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","4 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Llama 3-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 30 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"google-deepmind-harmful-manipulation-studies","name":"Google DeepMind harmful manipulation studies","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"google-deepmind-internal-cyber-benchmarks","name":"Google DeepMind internal cyber benchmarks","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"google-deepmind-scheming-honeypot-suite","name":"Google DeepMind scheming honeypot suite","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"google-landmarks-dataset-v2","name":"Google Landmarks Dataset v2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"google-re-bench-grb","name":"Google RE-Bench (GRB)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 3.7 Flash (#26)."],["scores.headroom","pass","The best is 27%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"google-scanned-objects","name":"Google Scanned Objects","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"google-universal-image-embedding-challenge","name":"Google Universal Image Embedding Challenge","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gorgias-ai-agent-benchmark","name":"Gorgias AI Agent Competitive Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on evals.gorgias.com: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gorilla-api-bench","name":"Gorilla APIBench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (6 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","pass","The best is 35.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"govreport","name":"GovReport","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gpbam","name":"GPBam","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (32 entries)."],["scores.models","pass","24 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 56 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gpqa-diamond","name":"GPQA Diamond","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries), Epoch AI’s runs (231 models), launch posts (254 scores) and system cards (168 scores)."],["scores.models","pass","408 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 97%, past 85% since June 2025."],["import.importer","pass","We read the board on kaggle.com: 20 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","7 of 129 pairs differ by more than 10 points, the largest for Magistral Small 1.2 (+22 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gpqa-extended","name":"GPQA Extended","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries) and system cards (3 scores)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in March 2024, for Claude 3 Haiku."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 48.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gpqa-main","name":"GPQA Main","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (66 entries), launch posts (3 scores) and system cards (5 scores)."],["scores.models","pass","66 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 80.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Stanford HELM: 62 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arithmetic-openai","name":"GPT-3 Arithmetic","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"word-scrambling-openai","name":"GPT-3 Word Scrambling","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gpu-puzzles","name":"GPU Puzzles","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5 mini. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 mini (#92)."],["scores.headroom","info","Saturated. The best is 90.5%, past 85% since August 2025."],["import.importer","pass","We read the board on app.primeintellect.ai: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gqa","name":"GQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","2 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for North Micro Vision Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"grade-like-a-human-os","name":"Grade Like a Human OS Dataset","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"grade-your-own-homework-can-llms-judge-each-other","name":"Grade Your Own Homework: Can LLMs Judge Each Other","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gradebench","name":"GradeBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 57.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on gradebench.com: 18 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"graduate-admissions-agent-benchmark","name":"Graduate Admissions Agent Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 9 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"graft","name":"GRAFT","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"grailqa","name":"GrailQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gram-gauging-realistic-agentic-misbehavior","name":"Gram (Gauging Realistic Agentic Misbehavior)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries) and system cards (1 score)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","pass","The best is 79.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gcdc","name":"Grammarly Corpus of Discourse Coherence","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 11 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"graphicdesignbench","name":"GraphicDesignBench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"graphwalks","name":"GraphWalks","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (25 scores) and system cards (12 scores)."],["scores.models","pass","29 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is Gemini 4 Carbon (#3)."],["scores.headroom","info","Saturated. The best is 99.96%, past 85% since March 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"graspbench","name":"GraspBench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gray-swan-art-benchmark","name":"Gray Swan Agent Red Teaming (ART) benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries) and system cards (14 scores)."],["scores.models","pass","23 models we track, plus 11 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on grayswan.ai: 11 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gray-swan-bio-eval-google","name":"Gray Swan Bio Eval","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gray-swan-cyber-eval-google","name":"Gray Swan Cyber Eval","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gray-swan-ipi-arena-benchmark","name":"Gray Swan IPI Arena benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries), launch posts (3 scores) and system cards (10 scores)."],["scores.models","pass","25 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gray-swan-jailbreak-arena","name":"Gray Swan Jailbreak Arena","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for o3-mini. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o3-mini (#124)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gray-swan-reward-hacking-bench","name":"Gray Swan Reward Hacking Bench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Beam 1.0."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gray-swan-shade-adaptive-attacker","name":"Gray Swan Shade (adaptive attacker)","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gray-swan-shade-knowledge-elicitation","name":"Gray Swan Shade knowledge elicitation through chat","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for Claude Fable 5.1."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gre-writing","name":"GRE Analytical Writing","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 2 years ago, in March 2024, for Claude 3 Opus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gre-physics","name":"GRE Physics","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gre-quantitative","name":"GRE Quantitative","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gre-verbal","name":"GRE Verbal","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"greek-legal-code","name":"Greek Legal Code","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (186 entries)."],["scores.models","pass","5 models we track, plus 181 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 56.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MTEB: 5 rows for models we track and 181 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"greekbarbench","name":"GreekBarBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","8 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for GPT-4.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 8 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"greekbarretrieval","name":"GreekBarRetrieval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","4 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for gpt-oss-120b."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is gpt-oss-120b (#126)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"greekmmlu","name":"GreekMMLU","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (73 entries)."],["scores.models","pass","34 models we track, plus 39 other entries on its leaderboard."],["scores.recent","info","The newest came 11 months ago, in December 2025, for Gemini 3 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Saturated. The best is 93.2%, past 85% since December 2025."],["import.importer","pass","We read the board on arxiv.org: 69 rows for models we track and 39 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"green-job-description-entities","name":"Green Job Description Entity Recognition","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"greptile-code-review-benchmark","name":"Greptile AI Code Review Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gretel-synthetic-pii-finance","name":"Gretel Synthetic PII Finance","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"grid-perception-reliability","name":"Grid Perception Reliability","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.7 Flash (#26)."],["scores.headroom","info","Saturated. The best is 99.1%, past 85% since December 2025."],["import.importer","pass","We read the board on kaggle.com: 3 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stech-grooming-the-machine","name":"Grooming the Machine","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arianna-stech.github.io: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"groundcocoa","name":"GroundCocoa","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","2 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in December 2023, for Gemini 1.0 Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 41.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"grounded-language-model-customer-arena-contextual","name":"Grounded Language Model Customer Evaluations (Contextual AI)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-grounding-rag-google","name":"Grounding robustness (RAG prompt injection)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 99%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"grounding-with-citations-unfactual-rate-google","name":"Grounding with citations, sentence unfactual rate","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"groundtruth-geology","name":"Groundtruth Dynamic Benchmarking (Geology Edition)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eigenform.ai: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"groundui","name":"GroundUI","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (13 entries) and launch posts (1 score)."],["scores.models","pass","12 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Amazon Nova Act."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","pass","The best is 80.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ltzheng.github.io: 11 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"group-binary-search-riedl","name":"Group Binary Search (Riedl)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Qwen3-235B-A22B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-235B-A22B-Instruct (#133)."],["scores.headroom","pass","The best is 75%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"groupassistbench","name":"GroupAssistBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gs-qa-2","name":"GS-QA2","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 1 entry at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gsm-infinite","name":"GSM-Infinite","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","14 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for DeepSeek-R1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-R1 (#122)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on infiniailab-gsm-infinite-leaderboard.hf.space: 20 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gsm-plus","name":"GSM-Plus","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (25 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","8 models we track, plus 19 other entries on its leaderboard."],["scores.recent","info","The newest came 12 months ago, in November 2025, for LFM2-1.2B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Saturated. The best is 85.6%, past 85% since June 2023."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 19 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gsm-symbolic","name":"GSM-Symbolic","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (25 entries) and launch posts (3 scores)."],["scores.models","pass","20 models we track, plus 8 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Granite 4.1 30B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Saturated. The best is 94.9%, past 85% since July 2024."],["import.importer","pass","We read the board on arxiv.org: 17 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gsm1k","name":"GSM1k","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","15 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","info","Saturated. The best is 96.6%, past 85% since February 2024."],["import.importer","pass","We read the board on labs.scale.com: 15 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gsm8k","name":"GSM8K","summary":{"pass":8,"warn":1,"fail":1,"info":2,"text":"8 of 10 checks pass, 1 warning, 1 problem"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (92 entries), launch posts (55 scores), system cards (71 scores) and claims Epoch AI collected (70 models)."],["scores.models","pass","208 models we track, plus 14 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Saturated. The best is 98.1%, past 85% since March 2023."],["import.importer","pass","We read the board on AGI-Eval: 98 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","2 of 15 pairs differ by more than 10 points, the largest for Gemma 2 9B (−19 pts)."],["consistency.issues","fail","1 open finding: Contamination."]]},{"id":"gsm8k-platinum-268","name":"GSM8K Platinum Initial 268","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","21 models we track, plus 10 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for GPT-4.1. That’s expected for a test replaced by GSM8K-Platinum."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.5 Pro (#104)."],["scores.headroom","info","Saturated. The best is 99.6%, past 85% since June 2024."],["import.importer","pass","We read the board on platinum-bench.csail.mit.edu: 23 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gsm8k-platinum","name":"GSM8K-Platinum","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries) and system cards (1 score)."],["scores.models","pass","9 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Apertus 70B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1 (#118)."],["scores.headroom","info","Saturated. The best is 99.8%, past 85% since October 2024."],["import.importer","pass","We read the board on platinum-bench.csail.mit.edu: 10 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gsm8k-v","name":"GSM8K-V","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (31 entries)."],["scores.models","pass","26 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Kimi K2.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 59.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 29 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gso-bench","name":"GSO-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries)."],["scores.models","pass","30 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 88.2%, past 85% since September 2026."],["import.importer","pass","We read the board on gso-bench.github.io: 32 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gsr-bench","name":"GSR-Bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","3 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gta-tool-agents","name":"GTA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","15 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in May 2024, for GPT-4o. That’s expected for a test replaced by GTA-2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 46.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on open-compass.github.io: 15 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gta-2","name":"GTA-2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","9 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 11.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gtbench","name":"GTBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-3.5 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 13 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gtfs-retrieval","name":"GTFS Retrieval","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on link.springer.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gtfs-semantics","name":"GTFS Semantics","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gto-wizard-benchmark","name":"GTO Wizard Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (98 entries)."],["scores.models","pass","6 models we track, plus 92 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on benchmark.gtowizard.com: 6 rows for models we track and 92 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gtzan","name":"GTZAN","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-aggregate-guesswork","name":"Guesswork (The Aggregate)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","22 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on theaggregate.ai: 22 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gui-knowledge-bench","name":"GUI Knowledge Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5 Chat."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 78.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on computer-use-agents.github.io: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gui-odyssey","name":"GUI Odyssey","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","3 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for InternVL2-Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 55%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gui-ceval-xiaomi","name":"GUI-CEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","6 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for MiMo-VL-7B-RL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 61.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openaccess.thecvf.com: 6 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"guiacat","name":"GuiaCat","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (320 entries)."],["scores.models","pass","35 models we track, plus 285 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for LFM2.5-1.2B-Thinking."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is gpt-oss-20b (#143)."],["scores.headroom","pass","The best is 69.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on euroeval.com: 39 rows for models we track and 285 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gulf-text-to-sql-bench","name":"Gulf Text-to-SQL Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","info","Saturated. The best is 96.7%, past 85% since August 2025."],["import.importer","pass","We read the board on kaggle.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gutbrainie-2025","name":"GutBrainIE 2025","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on hereditary.dei.unipd.it: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gutbrainie-2026","name":"GutBrainIE 2026","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on hereditary.dei.unipd.it: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gym-anything-rt","name":"Gym-Anything-RT","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 9.00%, under 85%, so it still separates models."],["import.importer","pass","We read the board on gym-anything.github.io: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"h2r-bench","name":"H2R-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","5 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for LTX-2.3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on rongdingyi.github.io: 5 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"habit-bench","name":"HABIT-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GLM-5.3-Flash (#53)."],["scores.headroom","pass","The best is 42%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 66 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hack-the-box-google","name":"Hack the Box challenge set (Google DeepMind)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 15.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hack-the-box-machines-meta","name":"Hack the Box Machines (Meta)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 13 months ago, in October 2025, for Code World Model (CWM). Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 41%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hack-verifiable-terminal-bench","name":"Hack-Verifiable Terminal Bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hackbench-meta","name":"HackBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hackerbench","name":"HackerBench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","pass","The best is 7.80%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hacksynth-overthewire","name":"HackSynth OverTheWire","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for Phi-3.5-MoE."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-70B (#177)."],["scores.headroom","pass","The best is 28.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hacksynth-picoctf","name":"HackSynth PicoCTF","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for Phi-3.5-MoE."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-70B (#177)."],["scores.headroom","pass","The best is 22.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hae-rae-bench","name":"HAE-RAE Bench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores) and system cards (2 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for Solar Open 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 86.2%, past 85% since July 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hae-rae-bench-1-0","name":"HAE-RAE Bench 1.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hagrid","name":"HAGRID","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"halachabench","name":"HalachaBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries)."],["scores.models","pass","28 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 28 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"halas","name":"HALAS","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","2 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in October 2024, for Whisper large-v3-turbo."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hallmarks-of-cancer","name":"Hallmarks of Cancer","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on repository.cam.ac.uk: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hallomtbench","name":"HalloMTBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"production-hallucinations-openai","name":"Hallucinations on production-representative prompts","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 11 months ago, in December 2025, for GPT-5.2. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"halludial","name":"HalluDial","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (21 entries)."],["scores.models","pass","9 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hallulens","name":"HalluLens","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"halluscoring-2026","name":"HalluScoring 2026","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hallusionbench","name":"HallusionBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries), launch posts (25 scores) and system cards (18 scores)."],["scores.models","pass","32 models we track, plus 17 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5 397B-A17B (#78)."],["scores.headroom","pass","The best is 73.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 17 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"halluworld","name":"HalluWorld","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 19.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on halluworld.ai: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hallym-womens-health-nursing-exam","name":"Hallym University Women's Health Nursing Exam","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on pmc.ncbi.nlm.nih.gov: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"halteval-prelim-meta","name":"HaltEval-prelim (Meta)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Code World Model (CWM). That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 94%, past 85% since September 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"halubench","name":"HaluBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","2 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in March 2024, for Claude 3 Haiku."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Sonnet (#187)."],["scores.headroom","pass","The best is 78.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"halueval","name":"HaluEval","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Nemotron 3 Nano 4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"haluevalaudio-1000","name":"HaluEvalAudio 1000","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"halumem","name":"HaluMem","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hanabi-agent-benchmark","name":"Hanabi Agent Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","19 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 19 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"handbook-md","name":"HANDBOOK.md","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries) and launch posts (1 score)."],["scores.models","pass","18 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Small."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 52.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on surgehq.ai: 17 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hands-visible","name":"Hands Visible","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"handworldbench","name":"HandWorldBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","1 model we track, plus 10 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on embodiedcity.github.io: 1 row for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hangman-bench","name":"Hangman Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hans","name":"HANS","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"happyworld-embodied","name":"HappyWorld Embodied","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on skyeval.com: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"happyworld-reconstruction","name":"HappyWorld Spatial Reconstruction","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","1 model we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on skyeval.com: 1 row for models we track and 8 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"happyworld-bench","name":"HappyWorld-Bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on skyeval.com: 14 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harbor-index","name":"Harbor-Index","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries) and launch posts (1 score)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 39.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tbench.ai: 18 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hard-capability-inhouse-minimax","name":"Hard Capability (MiniMax in-house)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for MiniMax-Text-01. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 64.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hard-negative-protein-binding-openai","name":"Hard-negative protein binding prediction","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 7.60%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hard12","name":"Hard12","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","11 models we track, plus 15 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 83.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ibragim.dev: 40 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hardest-sat-problems-vellum","name":"Hardest SAT Problems (Vellum)","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Claude Opus 4. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","info","Saturated. The best is 94%, past 85% since April 2025."],["import.importer","pass","We read the board on vellum.ai: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harmbench","name":"HarmBench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (4 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for TML-Interaction-Small. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-R1 (#130)."],["scores.headroom","info","Saturated. The best is 99%, past 85% since December 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harmeval","name":"HarmEval","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in December 2023, for Mistral-7B-Instruct-v0.2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harmful-content-defect-rate-microsoft","name":"Harmful content defect rate (Azure AI Evaluation SDK, Microsoft)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Phi-4-Reasoning-Vision-15B. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harmful-manipulation-efficacy-study-google","name":"Harmful manipulation efficacy study","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harmful-manipulation-propensity-evaluation-google","name":"Harmful manipulation propensity evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 3.7 Flash (#26)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"strategic-planning-red-teaming-google","name":"Harmful manipulation strategic planning red teaming","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harmfulness-on-held-out-prompts-anthropic","name":"Harmfulness scores on held-out prompts (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 3 years ago, in July 2023, for Claude 2. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude 2 (#192)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harmlessness-human-feedback-win-rate-anthropic","name":"Harmlessness human feedback win rate (Anthropic)","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 2 years ago, in October 2024, for Claude 3.5 Haiku. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude 3.5 Sonnet (October 2024) (#150)."],["scores.headroom","info","Saturated. The best is 87%, past 85% since March 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harmogen-r","name":"HARMOGEN-R","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harmony4d","name":"Harmony4D","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harness-leaderboard","name":"Harness Leaderboard","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harness-bench-fast","name":"harness-bench-fast","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 11 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harnessaudit-bench","name":"HarnessAudit-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Kimi K2.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on harnessaudit.github.io: 10 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harnessdev","name":"HarnessDev","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 67.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harnesseval-w","name":"HarnessEval-W","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","4 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on xbench.org: 4 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harnessopt-bench","name":"HarnessOpt-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on labs.scale.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harp-bench","name":"HARP-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in October 2024, for π0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 18.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on harp-bench.vercel.app: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harvest-rush-train","name":"Harvest Rush Training Evaluation","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-8B (#203)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harvestbench","name":"HarvestBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on compassionbench.com: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harvey-lab-legal-agent-benchmark","name":"Harvey LAB (Legal Agent Benchmark)","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (80 entries), launch posts (10 scores) and system cards (11 scores)."],["scores.models","pass","83 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 25.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 81 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 10 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harvey-lab-1-1","name":"Harvey LAB v1.1 (HLAB-Vals-1.1)","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (43 entries) and launch posts (2 scores)."],["scores.models","pass","42 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 27.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 42 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harvey-lab-aa","name":"Harvey LAB-AA v1.0","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Saturated. The best is 94.6%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"harvey-lab-aa-1-1","name":"Harvey LAB-AA v1.1","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (4 scores) and system cards (3 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 15%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hate-harassment-safety-eval-google","name":"Hate & Harassment Safety","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 92.7%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hateful-memes-challenge","name":"Hateful Memes Challenge","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on DrivenData: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hc3-finance-reranking","name":"HC3Reranking","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hcast","name":"HCAST","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on metr.org that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hcsu-bench","name":"HCSU-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","7 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 34.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hd-epic-vqa","name":"HD-EPIC VQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Qwen2.5-VL-7B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 28.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on codabench.org: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hdlbits","name":"HDLBits","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"head-qa","name":"HEAD-QA","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aghie.github.io: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"head-qa-2","name":"HEAD-QA v2","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-70B (#177)."],["scores.headroom","pass","The best is 83.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 20 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healbench-eval","name":"HealBench-Eval","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","2 models we track, plus 14 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"health-dynamic-conversation-eval-google","name":"Health dynamic conversation eval","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"health-localization-eval-google","name":"Health language and locale eval","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"health-static-understanding-eval-google","name":"Health static understanding eval","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthadvice","name":"HealthAdvice","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthagentbench","name":"HealthAgentBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 55%, under 85%, so it still separates models."],["import.importer","pass","We read the board on microsoft.github.io: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthagentbench-2","name":"HealthAgentBench V2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 57.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on microsoft.github.io: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthbench","name":"HealthBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries), launch posts (10 scores) and system cards (36 scores)."],["scores.models","pass","39 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 67.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openai.com: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthbench-consensus","name":"HealthBench Consensus","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries), launch posts (1 score) and system cards (25 scores)."],["scores.models","pass","26 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6 Luna."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 96.6%, past 85% since August 2024."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthbench-consensus-global-health","name":"HealthBench Consensus Global Health","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries) and system cards (2 scores)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on cdn.openai.com: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthbench-consensus-urgent","name":"HealthBench Consensus Urgent","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries) and system cards (2 scores)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on cdn.openai.com: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthbench-hard","name":"HealthBench Hard","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries), launch posts (7 scores) and system cards (28 scores)."],["scores.models","pass","29 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6 Luna."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 46.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openai.com: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthbench-hard-hallucinations","name":"HealthBench Hard Hallucinations","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on cdn.openai.com: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthbench-meta-openai","name":"HealthBench Meta","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for o3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthbench-professional","name":"HealthBench Professional","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries), launch posts (17 scores) and system cards (34 scores)."],["scores.models","pass","35 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 69.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openai.com: 7 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthbench-hallu-baichuan","name":"HealthBench-Hallu","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","5 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Baichuan-M3-235B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthbench-psych","name":"HealthBench-Psych v1","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","22 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 22 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthbench-psych-2","name":"HealthBench-Psych v2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","27 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 27 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthbench-psych-hard","name":"HealthBench-Psych-Hard","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","27 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 27 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"healthqa-br","name":"HealthQA-BR","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","14 models we track, plus 7 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for GPT-4.1. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4.1 (#138)."],["scores.headroom","info","Saturated. The best is 86.6%, past 85% since April 2025."],["import.importer","pass","We read the board on huggingface.co: 14 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hearingbench","name":"HearingBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 46%, under 85%, so it still separates models."],["import.importer","pass","We read the board on hearing-bench.adamholter.chatgpt.site: 57 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"heart-emotional-support","name":"HEART","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","13 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hebbia-financial-services-benchmark","name":"Hebbia Financial Services Benchmark","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 74.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on fireworks.ai: 10 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hebbia-financial-services-benchmark-2025","name":"Hebbia Financial Services Benchmark (2025)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dicta-hebrew-diacritization-test-corpora","name":"Hebrew Diacritization Test Corpora","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hebrew-llm-chat-leaderboard","name":"Hebrew LLM Chat Leaderboard","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","33 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GLM-5.3-Flash (#53)."],["scores.headroom","pass","The best is 74.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 33 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hedge-gym","name":"Hedge Gym","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hedge-bench","name":"Hedge-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"heim","name":"HEIM","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on crfm.stanford.edu: 26 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hellaswag","name":"HellaSwag","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries), launch posts (28 scores), system cards (39 scores) and claims Epoch AI collected (63 models)."],["scores.models","pass","113 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Saturated. The best is 97%, past 85% since November 2022."],["import.importer","pass","We read the board on OpenCompass: 16 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 5 pairs differs by more than 10 points, the largest for Gemma 3 27B (+17 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hellobench","name":"HelloBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (23 entries)."],["scores.models","pass","11 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for o1-mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 11 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"helm-arabic","name":"HELM Arabic","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries)."],["scores.models","pass","28 models we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Claude Opus 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on crfm.stanford.edu: 28 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"helm-arabic-enterprise","name":"HELM Arabic Enterprise","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (35 entries)."],["scores.models","pass","23 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on crfm.stanford.edu: 23 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"helm-capabilities","name":"HELM Capabilities","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (66 entries)."],["scores.models","pass","60 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on crfm.stanford.edu: 62 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"helm-classic","name":"HELM Classic","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (67 entries)."],["scores.models","pass","17 models we track, plus 50 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Llama 2-13B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-3.5 Turbo (#209)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on crfm.stanford.edu: 17 rows for models we track and 50 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"helm-lite","name":"HELM Lite","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (91 entries)."],["scores.models","pass","72 models we track, plus 19 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for DeepSeek-V3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3 (#141)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on crfm.stanford.edu: 72 rows for models we track and 19 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"helm-safety","name":"HELM Safety","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (81 entries)."],["scores.models","pass","75 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on crfm.stanford.edu: 81 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"helmet","name":"HELMET","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (60 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","17 models we track, plus 44 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Falcon-H1-34B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","pass","The best is 61.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on princeton-nlp.github.io: 100 rows for models we track and 44 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"helpbench","name":"HelpBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries) and system cards (1 score)."],["scores.models","pass","10 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","pass","The best is 84%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"art-helpfulness-violations-google","name":"Helpfulness violations (from ART)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hemingway-bench","name":"Hemingway-bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (46 entries)."],["scores.models","pass","44 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on surgehq.ai: 46 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"heptoolbench-1-2","name":"HEPToolBench 1.2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","29 models we track, plus 13 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Gemma 4 12B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.5 Flash (#44)."],["scores.headroom","pass","The best is 71.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 29 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"heq","name":"HeQ","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (201 entries)."],["scores.models","pass","88 models we track, plus 113 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemma 4 31B IT (#101)."],["scores.headroom","info","Saturated. The best is 87%, past 85% since September 2024."],["import.importer","pass","We read the board on huggingface.co: 89 rows for models we track and 113 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hermes-bench-nous","name":"Hermes Bench","summary":{"pass":9,"warn":2,"fail":0,"info":1,"text":"9 of 11 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Tencent Hy4."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 76.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on portal.nousresearch.com: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hermes-index-nous","name":"Hermes Index","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Tencent Hy4."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on portal.nousresearch.com: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hexmachina","name":"HexMachina","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hi4d","name":"Hi4D","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on yifeiyin04.github.io: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hiacc","name":"HiACC","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Qwen3-ASR-0.6B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on Srota-Conv: 1 row for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hidden-agenda-google","name":"Hidden agenda","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in July 2024, for Gemma 2 27B. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hidden-voices-audio-difficulty","name":"Hidden Voices Audio Difficulty","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hiddenmath","name":"HiddenMath","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (7 scores) and system cards (7 scores)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from July 2026, for DiffusionGemma 26B A4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 2.0 Pro (#140)."],["scores.headroom","pass","The best is 80.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hiddenmath-hard","name":"HiddenMath-Hard","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 80.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hiding-a-secret-password-anthropic","name":"Hiding a secret password (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hidrive","name":"HiDrive","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hiertext","name":"HierText","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"high-stakes-factuality-openai","name":"High Stakes factuality prompts","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Sol (#13)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"health-high-risk-scenarios-eval-google","name":"High-risk health scenarios eval","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 98.9%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"higher-difficulty-benign-request-evaluations-anthropic","name":"Higher-difficulty benign request evaluations (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Opus 4.7. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.7 (#31)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"higher-difficulty-harmful-request-evaluations-anthropic","name":"Higher-difficulty harmful request evaluations (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Opus 4.7. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.7 (#31)."],["scores.headroom","info","Saturated. The best is 99.4%, past 85% since February 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hil-bench-scale","name":"HiL-Bench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (37 entries)."],["scores.models","pass","37 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 79.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 37 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hindsight-neglect","name":"Hindsight Neglect","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 3 years ago, in March 2023, for GPT-4."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hipe-ocrepair-2026","name":"HIPE-OCRepair 2026","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on hipe-eval.github.io: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hipho","name":"HiPhO","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries), launch posts (3 scores) and system cards (1 score)."],["scores.models","pass","40 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","info","Saturated. The best is 90%, past 85% since August 2026."],["import.importer","pass","We read the board on phyarena.github.io: 39 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hirewatch","name":"HireWatch","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries)."],["scores.models","pass","16 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Claude Opus 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 16 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"histoires-morales","name":"Histoires Morales","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for Mistral-7B-Instruct-v0.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Mistral 7B v0.1 (#212)."],["scores.headroom","pass","The best is 54.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"historical-jailbreak-regression-openai","name":"Historical Jailbreak Regression","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for GPT-Rosalind-6."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"historical-systems-bugs","name":"Historical Systems Bugs","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hle-bio-chem-gold","name":"HLE Bio/Chem Gold","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hle-calibration","name":"HLE Calibration","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries), launch posts (1 score) and system cards (4 scores)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on scale.com: 4 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hle-extended-mercor","name":"HLE Extended","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","31 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 77.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mercor.com: 31 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hle-diamond","name":"HLE-Diamond","summary":{"pass":11,"warn":0,"fail":1,"info":0,"text":"11 of 12 checks pass, 1 problem"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries), launch posts (5 scores) and system cards (2 scores)."],["scores.models","pass","26 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 77.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on lastexam.ai: 44 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","fail","1 open finding: Leaked answers and reward hacking."]]},{"id":"hle-rolling","name":"HLE-Rolling","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hle-verified","name":"HLE-Verified","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries), launch posts (3 scores) and system cards (4 scores)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","pass","The best is 73.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hle-verified-text-full","name":"HLE-Verified (text-only Full)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","6 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 46.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 6 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hle-vl","name":"HLE-VL","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries), launch posts (3 scores) and system cards (2 scores)."],["scores.models","pass","11 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 52.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 14 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hls-eval","name":"HLS-Eval","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 23 months ago, in December 2024, for DeepSeek-V3. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since November 2024."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hm3d-mfmon","name":"HM3D-MFMON","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hm3dsem-walks","name":"HM3DSem-Walks","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on hovsg.github.io: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hmdb51","name":"HMDB51","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hmmt-2025","name":"HMMT 2025","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (53 entries), launch posts (48 scores) and system cards (25 scores)."],["scores.models","pass","93 models we track."],["scores.recent","pass","The newest is from August 2026, for Granite 4.2 30B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since July 2025."],["import.importer","pass","We read the board on MathArena: 60 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 14 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hmmt-2026","name":"HMMT 2026","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries), launch posts (18 scores) and system cards (7 scores)."],["scores.models","pass","45 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Small."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 98.5%, past 85% since November 2025."],["import.importer","pass","We read the board on MathArena: 31 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 5 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hmmt-2024","name":"HMMT February 2024","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hmmt-nov-2025","name":"HMMT Nov 2025","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries), launch posts (24 scores) and system cards (11 scores)."],["scores.models","pass","45 models we track."],["scores.recent","pass","The newest is from June 2026, for GLM-5.2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Saturated. The best is 99.2%, past 85% since July 2025."],["import.importer","pass","We read the board on matharena.ai: 23 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 5 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hmu-bench","name":"HMU-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.7 Flash (#26)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hold-or-fold-ai-under-pushback","name":"Hold or Fold: AI Under Pushback","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"holdings-individualtaxbench","name":"Holdings-IndividualTaxBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 76.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on evals.thriveholdings.com: 50 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"holoassist-mistake-detection","name":"HoloAssist Mistake Detection","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 1 entry at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"holobench","name":"HoloBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Gemini 2.0 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3-mini (#124)."],["scores.headroom","pass","The best is 59.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 28 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"homayouni-fontes-fjspt","name":"Homayouni and Fontes Flexible Job Shop with Transportation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"homeloop","name":"HomeLoop","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 61.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"honesty-human-feedback-win-rate-anthropic","name":"Honesty human feedback win rate (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 69%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"honeybench-v0-2-alpha","name":"HoneyBench v0.2-alpha","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"honeypot-exploitation-monitorability-openai","name":"Honeypot exploitation monitorability","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"honor-of-kings-bug-ai-root-cause-tencent","name":"Honor of Kings Bug AI Root Cause Analysis","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for Tencent Hy4."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 73.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on hunyuan.tencent.com: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"honor-of-kings-game-configuration-tencent","name":"Honor of Kings Game Configuration","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"horizonmath","name":"HorizonMath","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 11.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ewang26.github.io: 9 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hotpotqa","name":"HotpotQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (68 entries) and system cards (1 score)."],["scores.models","pass","1 model we track, plus 68 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Cohere Command A. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 92.1%, past 85% since April 2025."],["import.importer","pass","We read the board on hotpotqa.github.io: 68 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hotter-and-colder","name":"Hotter and Colder","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hourvideo","name":"HourVideo","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on raw.githubusercontent.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterprise-risk-mercor","name":"How frontier models miss risk in routine enterprise work","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"how2qa","name":"How2QA","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on VALUE: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hr-bench","name":"HR-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries) and launch posts (2 scores)."],["scores.models","pass","10 models we track, plus 8 other entries on its leaderboard."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Step 3.7 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Saturated. The best is 88.5%, past 85% since April 2026."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hrm8k","name":"HRM8K","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","11 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Solar Open 2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","info","Saturated. The best is 92.2%, past 85% since December 2025."],["import.importer","pass","We read the board on arxiv.org: 28 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hrm8k-ksm","name":"HRM8K-KSM","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for K-EXAONE 2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","info","Saturated. The best is 91.1%, past 85% since July 2025."],["import.importer","pass","We read the board on arxiv.org: 28 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"huayu-ai-cockpit-2026","name":"Huayu Awards AI Cockpit Evaluation 2026","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hubbench-blobfish","name":"HubBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on blobfish.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hf-speech-bench","name":"Hugging Face Speech Bench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"huggingface-agent-benchmark","name":"HuggingFace Agent Benchmark","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 3 years ago, in September 2023, for Qwen-1_8B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen-14B (#214)."],["scores.headroom","info","Saturated. The best is 93.5%, past 85% since September 2023."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"humaine","name":"HUMAINE","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (54 entries)."],["scores.models","pass","51 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 51 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"human-behavior-bench","name":"Human Behavior Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 11 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"human-chatgpt-comparison-corpus","name":"Human ChatGPT Comparison Corpus","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"human-feedback-win-rates-anthropic","name":"Human feedback win rates (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in October 2024, for Claude 3.5 Haiku. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"skills-il-human-or-ai","name":"Human or AI? (Skills IL)","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from October 2026, for Mistral Large 4."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 44.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on agentskills.co.il: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"human-pathogen-capabilities-test-securebio","name":"Human Pathogen Capabilities Test","summary":{"pass":10,"warn":2,"fail":0,"info":0,"text":"10 of 12 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (109 entries) and system cards (10 scores)."],["scores.models","pass","102 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 80.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on securebio.org: 98 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 4 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hpct-v2-securebio","name":"Human Pathogen Capabilities Test v2 (HPCT v2)","summary":{"pass":7,"warn":2,"fail":1,"info":2,"text":"7 of 10 checks pass, 2 warnings, 1 problem"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (74 entries)."],["scores.models","pass","72 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","pass","We read the board on securebio.org: 75 rows for models we track and 2 other entries at the last read."],["import.read","warn","November 2nd, 2026, 19 days before the data’s date."],["import.failed","fail","The read on November 18th, 2026 failed with “no chart matches /Model performance on HPCT-v2$/”. Pages show the rows from November 2nd, 2026."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"human-proximity-google","name":"Human proximity (Google DeepMind robotics)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemini Robotics ER 2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 93%, past 85% since July 2026."],["import.importer","pass","We read the board on storage.googleapis.com: 4 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"human-sourced-jailbreaks-openai","name":"Human Sourced Jailbreaks","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (8 scores)."],["scores.models","pass","8 models we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for o3. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o3 (#84)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"human-sxs-image-editing-google","name":"Human SxS image editing Elo (Google)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini Nano Banana 2.1."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"human-sxs-image-generation-google","name":"Human SxS image generation Elo (Google)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini Nano Banana 2.1."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"human-verification-gap","name":"Human Verification Gap","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","34 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 35 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"humanagencybench","name":"HumanAgencyBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries)."],["scores.models","pass","34 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 71.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 35 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"humanclaw-bench","name":"HumanCLAW-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","7 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 11.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on human-claw.github.io: 7 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"humaneval","name":"HumanEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (125 entries), launch posts (61 scores) and system cards (70 scores)."],["scores.models","pass","127 models we track, plus 79 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Saturated. The best is 98.8%, past 85% since March 2023."],["import.importer","pass","We read the board on EvalPlus: 46 rows for models we track and 79 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 18 pairs differs by more than 10 points, the largest for Mistral Large (−24 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"humaneval-infilling","name":"HumanEval Infilling","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Mercury Coder Small."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 84.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"humaneval-multi","name":"HumanEval Multi","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries), launch posts (4 scores) and system cards (3 scores)."],["scores.models","pass","5 models we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Kimi k1.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","pass","The best is 82.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 15 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"humaneval-plus","name":"HumanEval+","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (125 entries), launch posts (20 scores) and system cards (18 scores)."],["scores.models","pass","72 models we track, plus 79 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Granite 4.1 30B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Saturated. The best is 94.5%, past 85% since April 2024."],["import.importer","pass","We read the board on evalplus.github.io: 46 rows for models we track and 79 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"humaneval-cobol-cohere","name":"HumanEval-COBOL (Cohere)","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Cohere Command A."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 25.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"humaneval-v","name":"HumanEval-V","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (25 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","25 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for STEP3-VL-10B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 66%, under 85%, so it still separates models."],["import.importer","pass","We read the board on humaneval-v.github.io: 90 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"humaneval-x","name":"HumanEval-X","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 4 years ago, in April 2022, for GPT-NeoX-20B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 9.78%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"humanevalpack","name":"HumanEvalPack","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries) and system cards (3 scores)."],["scores.models","pass","3 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Cohere Command A."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen-14B (#214)."],["scores.headroom","pass","The best is 76.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"humanitys-last-exam","name":"Humanity's Last Exam","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (47 entries), launch posts (144 scores), system cards (73 scores) and claims Epoch AI collected (46 models)."],["scores.models","pass","182 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","5 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 74.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 55 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","8 large gaps of 21 pairs, each explained in our notes."],["consistency.issues","warn","1 finding disputed or fixed only in a copy: Wrong labels."]]},{"id":"humanitys-sixth-sense","name":"Humanity's Sixth Sense","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (38 entries) and launch posts (1 score)."],["scores.models","pass","38 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","8 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 75.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 38 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"humanoidbench","name":"HumanoidBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"humanoidtoolbench","name":"HumanoidToolBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","3 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Cosmos3-Edge."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 49.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snu-pi.github.io: 3 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hume-ai-overall-quality-index","name":"Hume AI Overall Quality Index","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hume-ai-voice-design-benchmark","name":"Hume AI Voice Design Benchmark","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hummusqa","name":"HumMusQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#112)."],["scores.headroom","pass","The best is 75.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"humor-in-ai-cartoon-captioning","name":"Humor in AI: Cartoon Captioning","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","2 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Claude 3 Opus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 73.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hungarian-national-high-school-exam","name":"Hungarian National High-School Exam","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","3 models we track, plus 16 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for DeepSeek LLM 67B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 2 (#192)."],["scores.headroom","pass","The best is 59%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 2 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hungarian-riddles-benchmark","name":"Hungarian Riddles Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash. That’s expected for a test replaced by CulturalRiddles."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 67.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hunyuan-mt-challenge-testset-tencent","name":"Hunyuan-MT Challenge Testset (Tencent)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Hunyuan-MT-7B. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"husky-hold-em-bench","name":"Husky Hold'em Bench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","warn","The record links a board on huskybench.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hw-bench","name":"HW-Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 67%, under 85%, so it still separates models."],["import.importer","pass","We read the board on hw-bench.com: 19 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hwe-bench-risc-v","name":"HWE Bench V1 (RISC-V CPU design)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","12 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on hwebench.com: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hwe-bench-risc-v-2","name":"HWE Bench V2 (RISC-V CPU design)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on hwebench.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hwe-bench-cae","name":"HWE-bench CAE","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 0.90%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cae-bench.svdailab.com: 8 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hwe-bench-rtl-repair","name":"HWE-Bench v1.0 (RTL bug repair)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 79.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on pku-liang.github.io: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hwe-bench-rtl-repair-1-1","name":"HWE-Bench v1.1 (RTL bug repair)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 99%, past 85% since August 2026."],["import.importer","pass","We read the board on pku-liang.github.io: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hx2dx-arena","name":"Hx2Dx ARENA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","pass","The best is 83.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy-backend-tencent","name":"Hy-Backend","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Tencent Hy3 preview. That’s expected for a test replaced by Hy-Backend 2.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 54.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy-backend-2-tencent","name":"Hy-Backend 2.0","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 35.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy-browsecomp-pro2-tencent","name":"Hy-BrowseComp-Pro2","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 56.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy-companybench-tencent","name":"Hy-CompanyBench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Tencent Hy3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 41.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy-companybench-2-tencent","name":"Hy-CompanyBench V2","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy-euler-pro-tencent","name":"Hy-Euler pro","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Tencent Hy3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 24.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy-finagentbench-tencent","name":"Hy-FinAgentBench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 79.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy-finmodelbench-tencent","name":"Hy-FinModelBench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Tencent Hy3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 69%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy-finmodelbench-2-tencent","name":"Hy-FinmodelBench v2","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 57%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy-lifesearch-tencent","name":"Hy-LifeSearch","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 49.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy-math-tencent","name":"Hy-Math","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Tencent Hy3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 60.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy-nomas-bench","name":"Hy-Nomas-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy-skillsworld-tencent","name":"Hy-SkillsWorld","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Tencent Hy3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 45.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy-swe-max-tencent","name":"Hy-SWE Max","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 64.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy-vibe-tencent","name":"Hy-Vibe Bench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Tencent Hy3 preview. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 50.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hy3d-bench","name":"HY3D-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hybriddeepresearch","name":"HybridDeepResearch","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-122B-A10B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5 397B-A17B (#78)."],["scores.headroom","pass","The best is 32.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snowflake-ai-research.github.io: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hydra-bench-snowflake","name":"HyDRA-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Claude Opus 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","pass","The best is 53.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 12 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hydrollm-benchmark","name":"HydroLLM-Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hype-trap","name":"Hype Trap","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hypersim","name":"Hypersim","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (9 scores) and system cards (1 score)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Qwen3.5-9B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5 397B-A17B (#78)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hypertask-minimax","name":"HyperTask","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for MiniMax-M2.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 67.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"i2b2-2006-smoking","name":"i2b2 2006 Smoking Challenge","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"i2b2-2008-obesity","name":"i2b2 2008 Obesity Challenge","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"i2b2-2009-medication","name":"i2b2 2009 Medication Extraction Challenge","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"i2b2-uthealth-2014-heart-disease","name":"i2b2/UTHealth 2014 Identifying Risk Factors for Heart Disease over Time","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iarpa-babel-speech-recognition","name":"IARPA Babel speech recognition","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iberobench","name":"IberoBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (33 entries)."],["scores.models","pass","5 models we track, plus 28 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Llama 3-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-8B (#206)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 5 rows for models we track and 28 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"icassp-song-aesthetics-2026","name":"ICASSP 2026 Automatic Song Aesthetics Evaluation Challenge","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aslp-lab.github.io: 19 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"icdar-maptext-2024","name":"ICDAR 2024 MapText Competition","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on rrc.cvc.uab.es: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"icdar-maptext-2025","name":"ICDAR 2025 MapText Competition","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on rrc.cvc.uab.es that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"icelandic-wikiqa","name":"Icelandic WikiQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (62 entries)."],["scores.models","pass","59 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 86.6%, past 85% since November 2026."],["import.importer","pass","We read the board on mideind-icelandic-llm-leaderboard.static.hf.space: 101 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"icelandic-winogrande","name":"Icelandic WinoGrande","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (93 entries)."],["scores.models","pass","70 models we track, plus 23 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 96.3%, past 85% since February 2024."],["import.importer","pass","We read the board on mideind-icelandic-llm-leaderboard.static.hf.space: 71 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"icho-2025","name":"IChO 2025","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3 Deep Think."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iclayout-bench","name":"ICLayout-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on lizhangmai.github.io: 18 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-icml-2025-experts","name":"ICML 2025 Experts Benchmark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","pass","The best is 81.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"icnale-edited-essays-scoring","name":"ICNALE Edited Essays","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"icnale-global-rating-archives","name":"ICNALE Global Rating Archives","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"icpc-world-finals-2025","name":"ICPC World Finals 2025","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Nemotron-Cascade 2 30B-A3B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ideataste","name":"IdeaTaste","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"idp-core-bench","name":"IDP Core Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","23 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Gemma 4 E2B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 89.6%, past 85% since February 2026."],["import.importer","pass","We read the board on idp-leaderboard.org: 23 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iemocap","name":"IEMOCAP","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"if-badcase-zai","name":"IF-Badcase","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for GLM-5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GLM-5 (#89)."],["scores.headroom","pass","The best is 83.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ifbench","name":"IFBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries), launch posts (69 scores) and system cards (43 scores)."],["scores.models","pass","83 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","info","Saturated. The best is 89.2%, past 85% since October 2026."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ifc-bench-2","name":"IFC-Bench v2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 65.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ONESTRUCTION: 20 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ifeval","name":"IFEval","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (66 entries), launch posts (100 scores) and system cards (88 scores)."],["scores.models","pass","178 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for NVIDIA Nemotron 3.5 Transcribe 30B-A4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.7-Max (#49)."],["scores.headroom","info","Saturated. The best is 98.9%, past 85% since July 2024."],["import.importer","pass","We read the board on Stanford HELM: 62 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 18 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ifeval-internal-mistral","name":"IFEval (Mistral internal version)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Mistral Small 3.2. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 87.4%, past 85% since June 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ifeval-ood","name":"IFEval-OOD","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries) and system cards (1 score)."],["scores.models","pass","5 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for OLMo 2 32B Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-70B (#177)."],["scores.headroom","pass","The best is 34.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ifevalcode","name":"IFEvalCode","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (42 entries)."],["scores.models","pass","25 models we track, plus 17 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-0.6B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 38.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 33 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ifmtbench","name":"IFMTBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","11 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen 3.6 35B-A3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 4 31B IT (#101)."],["scores.headroom","pass","The best is 81.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 11 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ifscale","name":"IFScale","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (22 entries)."],["scores.models","pass","20 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 68.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on distylai.github.io: 21 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ifstruct","name":"IFStruct","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries) and launch posts (2 scores)."],["scores.models","pass","13 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemma 4 31B IT (#101)."],["scores.headroom","info","Saturated. The best is 95.9%, past 85% since August 2025."],["import.importer","pass","We read the board on liquid.ai: 11 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ig-bench","name":"IG-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 27.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on visionxlab.github.io: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iheval","name":"IHEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","12 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark (#54)."],["scores.headroom","pass","The best is 82.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iit-madras-hindi-asr-challenge","name":"IIT Madras Hindi ASR Challenge","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on sites.google.com: 14 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iit-madras-hindi-tamil-english-asr","name":"IIT Madras Hindi-Tamil-English ASR Challenge","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on sites.google.com: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iitp-legal-english-hindi","name":"IITP-Legal","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (37 entries)."],["scores.models","pass","13 models we track, plus 24 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 13 rows for models we track and 24 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ilubench","name":"IlùBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2026."],["import.importer","pass","We read the board on uche.uuamni.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"image-generation-refusals-openai","name":"Image generation refusals","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for o3. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o3 (#84)."],["scores.headroom","info","Saturated. The best is 98%, past 85% since April 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"image-input-safety-evals-openai","name":"Image input safety evaluations","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1 (#118)."],["scores.headroom","info","Saturated. The best is 99%, past 85% since December 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"image-matching-challenge","name":"Image Matching Challenge","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on cs.ubc.ca: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"image-to-text-safety-google","name":"Image to Text Safety","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arena-image-to-webdev","name":"Image-to-WebDev Arena","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (55 entries)."],["scores.models","pass","55 models we track."],["scores.recent","pass","The newest is from November 2026, for Qwen4-Max."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arena.ai: 65 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"image2floorplan-bytedance","name":"Image2FloorPlan","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on seed.bytedance.com: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"image2struct","name":"Image2Struct","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","18 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in October 2024, for Claude 3.5 Sonnet (October 2024)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (October 2024) (#150)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on crfm.stanford.edu: 19 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imagecode","name":"ImageCoDe","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on mcgill-nlp.github.io: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imagejevbench-0-1-5","name":"ImageJevBench v0.1.5","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (50 entries)."],["scores.models","pass","4 models we track, plus 46 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on benchmarkheaven.com: 4 rows for models we track and 46 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imagejevbench-0-3-0","name":"ImageJevBench v0.3.0","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (64 entries)."],["scores.models","pass","4 models we track, plus 60 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on benchmarkheaven.com: 5 rows for models we track and 60 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"image-mining-zai","name":"ImageMining","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GLM-5V-Turbo."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 30.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imagenet-real","name":"ImageNet ReaL","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (1,266 entries)."],["scores.models","pass","1,266 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 1,266 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imagenet-1k","name":"ImageNet-1K","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1,266 entries)."],["scores.models","pass","1,266 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on timm: 1,266 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imagenet-a","name":"ImageNet-A","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1,266 entries)."],["scores.models","pass","1,266 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on timm: 1,266 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imagenet-c","name":"ImageNet-C","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 1 entry at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imagenet-r","name":"ImageNet-R","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1,266 entries)."],["scores.models","pass","1,266 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on timm: 1,266 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imagenet-sketch","name":"ImageNet-Sketch","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1,266 entries)."],["scores.models","pass","1,266 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on PyTorch Image Models: 1,266 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imagenet-ua","name":"ImageNet-UA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 10 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imagenetv2","name":"ImageNetV2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (67 entries)."],["scores.models","pass","67 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 67 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imb","name":"IMB","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Gemma 2 2B IT."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 71%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imc-2025","name":"IMC 2025","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Saturated. The best is 94.5%, past 85% since June 2025."],["import.importer","pass","We read the board on MathArena: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imdb-pt","name":"IMDb-PT","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imgedit-bench","name":"ImgEdit-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries) and system cards (1 score)."],["scores.models","pass","4 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Nova 2 Omni Preview."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imgmt","name":"ImgMT","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 3 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"immigration-bench","name":"Immigration Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","25 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 75.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 26 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imo-2025","name":"IMO 2025","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries), launch posts (8 scores) and system cards (6 scores)."],["scores.models","pass","12 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 83.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MathArena: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imo-2026","name":"IMO 2026","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imo-shortlist-2024","name":"IMO Shortlist 2024","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for DeepSeekMath-V2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 27%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imo-answerbench","name":"IMO-AnswerBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries), launch posts (33 scores) and system cards (19 scores)."],["scores.models","pass","46 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Spark-X2.6-4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.7-Max (#49)."],["scores.headroom","info","Saturated. The best is 91%, past 85% since January 2026."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imo-gradingbench","name":"IMO-GradingBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for Gemini 2.5 Deep Think."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 54%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imo-leanproofbench","name":"IMO-LeanProofBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 11.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on imobench.github.io: 1 row for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"imo-proofbench","name":"IMO-ProofBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries), launch posts (3 scores) and system cards (4 scores)."],["scores.models","pass","26 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Intern-S2-397B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 Pro (#15)."],["scores.headroom","info","Saturated. The best is 88.1%, past 85% since April 2026."],["import.importer","pass","We read the board on imobench.github.io: 21 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"implicit-intelligence-labelbox","name":"Implicit Intelligence","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Claude Opus 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Pro (#19)."],["scores.headroom","pass","The best is 53.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labelbox.com: 30 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"impossible-tasks-anthropic","name":"Impossible tasks (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (11 scores)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"impossible-tasks-eval-google","name":"Impossible Tasks Evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 93.8%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"impossiblebench","name":"ImpossibleBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for Muse Spark."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark (#54)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"impossiblemlebench-uk-aisi","name":"ImpossibleMLEBench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"improofbench","name":"IMProofBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 49.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on improofbench.math.ethz.ch: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"in-context-data-exfiltration-visual-browser-openai","name":"In-context data exfiltration, visual browser","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for ChatGPT agent. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 78%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdm-in-house-ctf","name":"In-house CTF suite (Phuong et al. 2024)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries) and system cards (4 scores)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 84.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"in22-gen","name":"IN22-Gen","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (192 entries)."],["scores.models","pass","5 models we track, plus 187 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 99.5%, past 85% since March 2026."],["import.importer","pass","We read the board on MTEB: 5 rows for models we track and 187 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inappropriate-image-prompts","name":"Inappropriate Image Prompts","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inas-hypothesis-classification","name":"INAS hypothesis classification","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inas-hypothesis-span-prediction","name":"INAS hypothesis span prediction","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inaturalist-2017","name":"iNaturalist 2017","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inaturalist-2018","name":"iNaturalist 2018","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inaturalist-2021","name":"iNaturalist 2021","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","21 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on SOTA2 Research: 21 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inaturalist-sounds","name":"iNaturalist Sounds","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"incident-arena","name":"Incident Arena","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 64.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on incident-arena.com: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"include","name":"INCLUDE","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries), launch posts (19 scores) and system cards (14 scores)."],["scores.models","pass","34 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 89.8%, past 85% since February 2026."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"include-indian-sign-language","name":"INCLUDE Indian Sign Language Recognition","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"incompebench","name":"IncompeBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"disallowed-and-sensitive-content-openai","name":"Incorrect behavior rate on disallowed and sensitive content","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 3 years ago, in March 2023, for GPT-4. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"indic-diarbench","name":"Indic DiarBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","2 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"indic-kcc-agri-advisory","name":"Indic-KCC-Agri-Advisory-Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","9 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V4-Flash (#87)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"indic-context-eval","name":"IndicContextEval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 21 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"indicgenbench","name":"IndicGenBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries), launch posts (2 scores) and system cards (4 scores)."],["scores.models","pass","18 models we track."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 14 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"indicifeval","name":"IndicIFEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","16 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 82.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 16 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"indicipr-qa","name":"IndicIPR-QA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","3 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Qwen2.5-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-8B (#203)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"indiclid-benchmark","name":"IndicLID benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (195 entries)."],["scores.models","pass","4 models we track, plus 191 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 95.1%, past 85% since June 2025."],["import.importer","pass","We read the board on MTEB: 4 rows for models we track and 191 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"indicparam","name":"IndicParam","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries)."],["scores.models","pass","19 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Claude Haiku 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 45%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 19 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"indicqa","name":"IndicQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"indicxnli","name":"IndicXNLI","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-actor-critic-google","name":"Indirect prompt injection, Actor Critic attack","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-beam-search-google","name":"Indirect prompt injection, Beam Search attack","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-tap-google","name":"Indirect prompt injection, TAP attack","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"indivibe","name":"IndiVibe","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Sarvam 105B. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 90%, past 85% since March 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"indommlu","name":"IndoMMLU","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries), launch posts (1 score) and system cards (4 scores)."],["scores.models","pass","9 models we track, plus 19 other entries on its leaderboard."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for Qwen2.5-14B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 69.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 5 rows for models we track and 19 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"indonli","name":"IndoNLI","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inductive-arithmetic-pattern-reasoning","name":"Inductive Arithmetic Pattern Reasoning","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2025."],["import.importer","pass","We read the board on kaggle.com: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"industrybench","name":"IndustryBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Qwen 3.6 Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 69.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 30 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ineqmath","name":"IneqMath","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (44 entries)."],["scores.models","pass","34 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for DeepSeek-V3.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 47%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 42 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inferencebench","name":"InferenceBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (44 entries)."],["scores.models","pass","26 models we track, plus 18 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","5 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","pass","We read the board on inferencebench.ai: 53 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inferenceeval-xai","name":"InferenceEval","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","pass","The best is 46.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"infihall-bench","name":"InfiHall-Bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","3 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Janus-Pro-7B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"infinite-generations-mistral","name":"Infinite Generations (Mistral internal)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Mistral Small 3.2. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"infinitebench-en-mc","name":"InfiniteBench/En.MC","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries), launch posts (1 score) and system cards (3 scores)."],["scores.models","pass","4 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","pass","The best is 83.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"infinitebench-en-qa","name":"InfiniteBench/En.QA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and system cards (3 scores)."],["scores.models","pass","4 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"infobench","name":"InFoBench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries) and system cards (1 score)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Cohere Command A. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","info","Saturated. The best is 94.9%, past 85% since November 2023."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"infographic-factuality-google","name":"Infographic factuality (Google)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini Nano Banana 2.1."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"infographicvqa","name":"InfographicVQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries), launch posts (15 scores) and system cards (18 scores)."],["scores.models","pass","26 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for North Micro Vision Instruct."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K2.5 (#76)."],["scores.headroom","info","Saturated. The best is 92.6%, past 85% since February 2025."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"information-extraction-bytedance","name":"Information Extraction (ByteDance Seed in-house)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Seed1.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 52%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"information-processing-bytedance","name":"Information Processing (ByteDance Seed in-house)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Seed1.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 53.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"infrabench-mi35x","name":"InfraBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"infrathon","name":"Infrathon","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on infrathonx.github.io: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"injongo","name":"INJONGO","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","5 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for Llama 3.3 70B Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#167)."],["scores.headroom","pass","The best is 65.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 5 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"innovationeval","name":"InnovationEval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"input-hallucinations-anthropic","name":"Input hallucinations (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Opus 4.7. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inquire-fullrank","name":"INQUIRE-Fullrank","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on inquire-benchmark.github.io: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inquire-rerank","name":"INQUIRE-Rerank","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","9 models we track, plus 14 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on inquire-benchmark.github.io: 9 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inquisitive","name":"INQUISITIVE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"insar-snow-axis-spatial","name":"InSAR Regional Snow Mapping (Axis Spatial)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","16 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since October 2025."],["import.importer","pass","We read the board on Axis Spatial: 16 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"insctrl-tencent","name":"InsCtrl (Tencent)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Hunyuan-A13B-Instruct. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 71.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"insider-trading-apollo","name":"Insider Trading Demo","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for GPT-3.5 Turbo Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"insight-consolidation-v1","name":"Insight Consolidation v1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inspect-india-evals","name":"Inspect India Evals","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"instinct-next-edit","name":"Instinct Next Edit","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"instruction-following-inhouse-minimax","name":"Instruction Following (MiniMax in-house)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for MiniMax-Text-01. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 46.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"helpfulness-instruction-following-google","name":"Instruction Following (safety evaluation)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"instruction-hierarchy-attacks-openai","name":"Instruction hierarchy attacks","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (10 scores)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6 Luna."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 99.99%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"instruction-hierarchy-openai","name":"Instruction Hierarchy Evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"instructir","name":"InstructIR","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"instructttseval","name":"InstructTTSEval","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Qwen3-TTS-12Hz-1.7B-CustomVoice."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 83%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"instrument-reading-google","name":"Instrument reading (Google DeepMind)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and launch posts (2 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemini Robotics ER 2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 93%, past 85% since April 2026."],["import.importer","pass","We read the board on storage.googleapis.com: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 2 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"instrumental-convergence-evals-google","name":"Instrumental convergence evals","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"instrumentaleval","name":"InstrumentalEval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","3 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for o3-mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iuk-1-3","name":"Instrumentation Universal Knowledge v1.3","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Grok 4.3. That’s expected for a test replaced by Instrumentation Universal Knowledge v1.5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 93.3%, past 85% since April 2025."],["import.importer","pass","We read the board on huggingface.co: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iuk-1-5","name":"Instrumentation Universal Knowledge v1.5","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Grok 4.3. That’s expected for a test replaced by Instrumentation Universal Knowledge v1.6 refresh."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 86.4%, past 85% since April 2025."],["import.importer","pass","We read the board on relayforge.tools: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iuk-1-6","name":"Instrumentation Universal Knowledge v1.6 refresh","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","25 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 93.3%, past 85% since February 2026."],["import.importer","pass","We read the board on relayforge.tools: 25 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"insurance-agent-benchmark-cooper","name":"Insurance Agent Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","17 models we track, plus 17 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","pass","The best is 77.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on askcooper.ai: 34 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"insurebench-broader-comparison","name":"InsureBench Decision Models (Broader Comparison)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","33 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on insurebench.insure-thing.com: 33 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"insurebench-multifunctional-llms","name":"InsureBench Multi-functional LLMs","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from June 2026, for Nemotron 3 Ultra."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 96.3%, past 85% since August 2025."],["import.importer","pass","We read the board on insurebench.insure-thing.com: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"intelligentbench-bytedance","name":"IntelligentBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for BAGEL-7B-MoT."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"intention-recognition-bytedance","name":"Intention Recognition (ByteDance Seed in-house)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Seed1.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 63.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"intentionally-taking-actions-very-rarely-anthropic","name":"Intentionally taking actions very rarely (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inter-gps","name":"Inter-GPS (Geometry3K)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 20 months ago, in March 2025, for Phi-4-multimodal. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 48.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"interactivebench","name":"InteractiveBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","pass","The best is 41.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on meta-interactiveworld.github.io: 12 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"interactive-benchmarks-logic","name":"InteractiveBench Logic","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 30.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on interactivebench.org: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"interactive-benchmarks-math","name":"InteractiveBench Math","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 76.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on interactivebench.org: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"interactive-benchmarks-poker","name":"InteractiveBench Texas Hold’em","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on interactivebench.org: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"interactive-benchmarks-trust","name":"InteractiveBench Trust Game","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on interactivebench.org: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"interactscience","name":"InteractScience","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries)."],["scores.models","pass","27 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 27 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"intercode-ctf","name":"InterCode-CTF","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries) and system cards (4 scores)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Claude Opus 4.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","info","Saturated. The best is 98.7%, past 85% since April 2025."],["import.importer","pass","We read the board on ukgovernmentbeis.github.io: 2 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"internal-factuality-evals-openai","name":"Internal adversarially-designed factuality evaluations","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 7.60%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"internal-agentic-coding-evaluation-anthropic","name":"Internal agentic coding evaluation (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 2 years ago, in October 2024, for Claude 3.5 Haiku. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude 3.5 Sonnet (October 2024) (#150)."],["scores.headroom","pass","The best is 78%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"internal-ai-research-evaluation-suite-2-anthropic","name":"Internal AI research evaluation suite 2","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Mythos Preview. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"api-instruction-following-openai","name":"Internal API instruction following (OpenAI)","summary":{"pass":9,"warn":2,"fail":0,"info":1,"text":"9 of 11 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (8 entries) and launch posts (4 scores)."],["scores.models","pass","10 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5 mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 mini (#92)."],["scores.headroom","pass","The best is 65.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openai.com: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"internal-codebase-monitor-evasion-google","name":"Internal codebase monitor evasion eval","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"internal-codebase-sabotage-propensity-anthropic","name":"Internal codebase sabotage propensity (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"internal-computer-use-safety-openai","name":"Internal computer use safety benchmark (OpenAI)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"internal-finance-modeling-moonshot","name":"Internal Finance Modeling (Moonshot)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"internal-financial-research-moonshot","name":"Internal Financial Research (Moonshot)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"internal-model-use-survey-anthropic","name":"Internal model use survey (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Mythos Preview. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 50%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"internal-research-debugging-openai","name":"Internal Research Debugging Evaluation","summary":{"pass":7,"warn":0,"fail":0,"info":3,"text":"All 7 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (8 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 81.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"internal-vulnerability-research-google","name":"Internal vulnerability research","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","pass","The best is 72%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"internbootcamp","name":"InternBootcamp","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for DeepSeek-R1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-R1 (#122)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"intervalqa","name":"IntervalQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 12 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"into-the-unknown","name":"Into the Unknown","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 0%, under 85%, so it still separates models."],["import.importer","pass","We read the board on prometheus-eval.github.io: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"intphys-2","name":"IntPhys 2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark Open (#43)."],["scores.headroom","pass","The best is 68.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"intphys2-timing","name":"IntPhys2-Timing","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","pass","The best is 77.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"introductory-sommelier-theory","name":"Introductory Sommelier (theory knowledge)","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 3 years ago, in March 2023, for GPT-4. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Saturated. The best is 92%, past 85% since March 2023."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"invalsi-italian-llm-benchmark-crisp","name":"INVALSI Italian LLM Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries)."],["scores.models","pass","31 models we track, plus 14 other entries on its leaderboard."],["scores.recent","info","The newest came 20 months ago, in March 2025, for GPT-4o. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Saturated. The best is 92.2%, past 85% since February 2024."],["import.importer","pass","We read the board on huggingface.co: 31 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"invented-o-clock","name":"Invented O'Clock","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","info","Saturated. The best is 97.8%, past 85% since February 2026."],["import.importer","pass","We read the board on kaggle.com: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inverse-ifeval","name":"Inverse IFEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (21 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","22 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 80.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 19 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"inverse-scaling-prize-tasks","name":"Inverse Scaling Prize Tasks","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 4 years ago, in April 2022, for PaLM (540B)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 78.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Wei et al.: 10 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"investment-banking-modeling-openai","name":"Investment Banking Modeling Tasks (OpenAI)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for GPT-5.5 Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.5 Pro (#15)."],["scores.headroom","info","Saturated. The best is 88.6%, past 85% since March 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"investors-last-exam","name":"Investor's Last Exam","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"invoice-approval","name":"Invoice Approval (Ali Siala)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-ioi","name":"IOI (Vals)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (55 entries)."],["scores.models","pass","55 models we track."],["scores.recent","pass","The newest is from August 2026, for Muse Spark 1.2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 91.7%, past 85% since July 2026."],["import.importer","pass","We read the board on vals.ai: 62 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ioi-2025","name":"IOI 2025","summary":{"pass":3,"warn":0,"fail":1,"info":6,"text":"3 of 4 checks pass, 1 problem"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","fail","Unverified. We haven’t checked it against a primary source yet."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from June 2026, for Nemotron 3 Ultra."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Nemotron 3 Ultra (#85)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-ioi-2","name":"IOI v2 (Vals)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (45 entries)."],["scores.models","pass","45 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since July 2026."],["import.importer","pass","We read the board on vals.ai: 46 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iol-ai-2026","name":"IOL-AI 2026","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (48 entries)."],["scores.models","pass","48 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on iolai.org: 48 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iosworld","name":"iOSWorld","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 51.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on iosworld.io: 12 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ipho-2025","name":"IPhO 2025","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 79.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ipi-bench","name":"IPI Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on ipibench.generalanalysis.com: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iptc-test","name":"IPTC-test","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","11 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 11 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iris-safety-evaluations-reflection","name":"Iris Safety Evaluations","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Beam 1.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Beam 1.0 (#67)."],["scores.headroom","info","Saturated. The best is 88.9%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"irokobench","name":"IrokoBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","11 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","pass","The best is 59%, under 85%, so it still separates models."],["import.importer","pass","We read the board on 9jabench.com: 22 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ironbench","name":"ironbench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5-9B (#128)."],["scores.headroom","pass","The best is 69.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cezman.github.io: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ironclad","name":"IronClad","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"irregular-frontiercyber-and-atomic-challenges","name":"Irregular FrontierCyber and Atomic Challenges","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Muse Spark 1.1."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.1 (#48)."],["scores.headroom","pass","The best is 70%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"irrelevant-instructions-text-browser-openai","name":"Irrelevant instructions, text-based web browser","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for ChatGPT agent. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 99.5%, past 85% since July 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"irrelevant-instructions-visual-browser-openai","name":"Irrelevant instructions, visual browser","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for ChatGPT agent. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 95%, past 85% since July 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"isa-bench","name":"ISA-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","10 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Granite 4.1 8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 4 31B IT (#101)."],["scores.headroom","pass","The best is 60.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"isaid","name":"iSAID","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"isarcasmeval","name":"iSarcasmEval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ishigaki-ids-bench","name":"Ishigaki-IDS-Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 63%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ONESTRUCTION: 20 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"isizulu-second-language-learner-speech","name":"IsiZulu Second Language Learner Speech Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"islamiceval-2025-quotation-correction","name":"IslamicEval 2025 Quotation Correction","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on codabench.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"islamiceval-2025-span-identification","name":"IslamicEval 2025 Quotation Span Identification","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on codabench.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"islamiceval-2025-quotation-validation","name":"IslamicEval 2025 Quotation Validation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on codabench.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"islamiceval-2025-quran-hadith-retrieval","name":"IslamicEval 2025 Quran and Hadith Passage Retrieval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codabench.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"islamicfaithqa","name":"IslamicFaithQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","7 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for gpt-oss-20b."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-14B (#139)."],["scores.headroom","pass","The best is 21.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 7 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"islamtrust","name":"IslamTrust","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"italian-administrative-casehold","name":"Italian-Administrative-CaseHold","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (192 entries)."],["scores.models","pass","4 models we track, plus 188 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 68.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mteb-leaderboard-backend.hf.space: 4 rows for models we track and 188 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"itbench","name":"ITBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Claude Opus 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 58.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 13 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"itbench-aa","name":"ITBench-AA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"itsmbench","name":"ITSMBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","14 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on enterpriseworlds.vibrantlabs.com: 15 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"itzi-roleplay-local-robustness","name":"Itzi Roleplay Benchmark: Local Robustness","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries)."],["scores.models","pass","36 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on itzi.app: 36 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ivista-voice-interaction-2023-revised","name":"IVISTA Voice Interaction, revised 2023 protocol","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ivo-micro1-contractbench","name":"Ivo-micro1 ContractBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 64%, under 85%, so it still separates models."],["import.importer","pass","We read the board on micro1.ai: 8 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"iwslt-2017-ted-translation","name":"IWSLT 2017 TED Translation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ja-vlm-bench-in-the-wild","name":"JA-VLM-Bench-In-the-Wild","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for llava-v1.6-mistral-7b."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jailbreak-augmented-examples-openai","name":"Jailbreak Augmented Examples","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for o3-mini. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1 (#118)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jailbreakbench","name":"JailbreakBench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Falcon H1R 7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-3.5 Turbo (#201)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on jailbreakbench.github.io: 48 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jailtrickbench","name":"JailTrickBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on proceedings.neurips.cc: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jam-alt","name":"Jam-ALT","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on MIREX: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jama-clinical-challenge","name":"JAMA Clinical Challenge (JesseLiu)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jamc-qa","name":"JamC-QA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","2 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Qwen2.5-72B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","pass","The best is 57.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jamendolyrics","name":"JamendoLyrics","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 1 entry at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jamendolyrics-multilang","name":"JamendoLyrics MultiLang","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jamp","name":"Jamp","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"janli","name":"JaNLI","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"japanese-llm-leaderboard-lm-evaluation-harness","name":"Japanese LLM Leaderboard (lm-evaluation-harness)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jnde-2024","name":"Japanese National Dental Examination 2024","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 22 months ago, in January 2025, for Gemini 2.0 Flash Thinking. That’s expected for a test replaced by Japanese National Dental Examination 2025."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 81.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on jds.ads.org.tw: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jnde-2025","name":"Japanese National Dental Examination 2025","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jneot-2025","name":"Japanese National Examination for Occupational Therapists 2025","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for o3. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","info","Saturated. The best is 88.4%, past 85% since April 2025."],["import.importer","pass","We read the board on link.springer.com: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jsn-nephrology-board-renewal-2014-2023","name":"Japanese Nephrology Board Renewal Questions 2014 to 2023","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for o1-pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 81.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on frontiersin.org: 1 row for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"japanese-tort-case-dataset","name":"Japanese Tort-case Dataset","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"japanese-english-bilingual-corpus-of-wikipedias-kyoto-articles","name":"Japanese-English Bilingual Corpus of Wikipedia’s Kyoto Articles","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jaqket","name":"JAQKET","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jarvis-bench-voicearena","name":"Jarvis Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Live."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on voicearena.com: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jasmine-medical-simplification","name":"JASMINE","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jawaher","name":"Jawaher","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jblimp","name":"JBLiMP","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jcola","name":"JCoLA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","2 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-3.5 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","pass","The best is 54.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jeem","name":"JEEM","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jef-bench-meta","name":"JEF-Bench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jemhopqa","name":"JEMHopQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jester-gesture-recognition","name":"Jester gesture recognition","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (33 entries)."],["scores.models","pass","33 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on openaccess.thecvf.com: 33 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jethics","name":"JETHICS","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#161)."],["scores.headroom","pass","The best is 71.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jevbench","name":"JevBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (345 entries)."],["scores.models","pass","6 models we track, plus 339 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Saturated. The best is 98.4%, past 85% since July 2026."],["import.importer","pass","We read the board on benchmarkheaven.com: 7 rows for models we track and 339 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jfbench","name":"JFBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","6 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 27B (#59)."],["scores.headroom","pass","The best is 48.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jfinbench","name":"JFinBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 97.3%, past 85% since June 2026."],["import.importer","pass","We read the board on pub.sakana.ai: 31 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jfinqa","name":"JFinQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jglue","name":"JGLUE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jgpqa","name":"JGPQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jhumaneval","name":"JHumanEval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jigsaw-toxic-comment-classification","name":"Jigsaw Toxic Comment Classification","summary":{"pass":3,"warn":3,"fail":0,"info":4,"text":"3 of 6 checks pass, 3 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Nova 2.0 Pro Preview."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 57.8%, under 85%, so it still separates models."],["import.importer","warn","The record links a board on kaggle.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jiminy-cricket","name":"Jiminy Cricket","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jmmlu","name":"JMMLU","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for Qwen2.5-14B. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 80.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jmultiwoz-tc","name":"JMultiWOZ-TC","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jobbench","name":"JobBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries), launch posts (18 scores) and system cards (6 scores)."],["scores.models","pass","31 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 75.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on job-bench.github.io: 16 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jobbert-job-title-normalization","name":"JobBERT Job Title Normalization","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"joi-bench-task","name":"Joi-Bench-Task","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jointavbench","name":"JointAVBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","9 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#112)."],["scores.headroom","pass","The best is 75.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-joke-arena","name":"Joke Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","25 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","7 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 27 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jorgpt-student-answer-grading","name":"JorGPT Student Answer Grading","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash-Lite."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on zenodo.org: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jsem","name":"JSeM","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jsick","name":"JSICK","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (207 entries)."],["scores.models","pass","4 models we track, plus 203 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 89.4%, past 85% since June 2025."],["import.importer","pass","We read the board on mteb-leaderboard-backend.hf.space: 4 rows for models we track and 203 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jsonschemabench","name":"JSONSchemaBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","5 models we track, plus 10 other entries on its leaderboard."],["scores.recent","info","The newest came 20 months ago, in March 2025, for GPT-4o. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","info","Saturated. The best is 91.4%, past 85% since March 2025."],["import.importer","pass","We read the board on docs.google.com: 5 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jsplib","name":"JSPLIB","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"atla-judge-arena","name":"Judge Arena (Atla)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","17 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 17 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"judge-bait","name":"Judge Bait","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","11 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2025."],["import.importer","pass","We read the board on kaggle.com: 12 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"judgebench","name":"JudgeBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (43 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","15 models we track, plus 30 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Llama-3.1-Nemotron-Ultra-253B-v1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3-mini (#124)."],["scores.headroom","pass","The best is 80.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 18 rows for models we track and 30 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"judgemark-v2-1","name":"Judgemark v2.1","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (47 entries)."],["scores.models","pass","46 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini. That’s expected for a test replaced by Judgemark v4."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eqbench.com: 46 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"judgemark-v4","name":"Judgemark v4","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","41 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eqbench.com: 41 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"judgerbench-opencompass","name":"JudgerBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","3 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in September 2024, for Qwen2.5-72B. That’s expected for a test replaced by JudgerBench V2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 3 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"judgerbench-v2","name":"JudgerBench V2","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","2 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-235B-A22B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-235B-A22B (#131)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"judgmentbench","name":"JudgmentBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.6 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Saturated. The best is 89%, past 85% since May 2026."],["import.importer","pass","We read the board on arxiv.org: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"justiceaxis","name":"JusticeAxis","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","6 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","pass","The best is 84.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"k-bench-mental-health-safety","name":"K-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","28 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 99%, past 85% since July 2024."],["import.importer","pass","We read the board on k-bench.ai: 111 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"k-browsecomp","name":"K-BrowseComp","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","8 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 45.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"k-metbench","name":"K-MetBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (65 entries)."],["scores.models","pass","43 models we track, plus 22 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Saturated. The best is 93.7%, past 85% since November 2025."],["import.importer","pass","We read the board on kmetbench.github.io: 49 rows for models we track and 22 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"k-mhas","name":"K-MHaS","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"k-qa","name":"K-QA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in June 2023, for GPT-4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","pass","The best is 67.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 4 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"k-sort-arena","name":"K-Sort Arena","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries)."],["scores.models","pass","5 models we track, plus 31 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 5 rows for models we track and 31 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"k-viscuit","name":"K-Viscuit","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","12 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for EXAONE 4.5 33B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","info","Saturated. The best is 89.5%, past 85% since March 2025."],["import.importer","pass","We read the board on arxiv.org: 19 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"k12-bench","name":"K12-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","6 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 31B IT."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 57.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 6 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"k12-pebench","name":"K12-PEBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","6 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Qwen2.5-VL-72B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 63.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"k12vista","name":"K12Vista","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (29 entries)."],["scores.models","pass","26 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Qwen2.5-VL-32B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3-mini (#124)."],["scores.headroom","pass","The best is 55.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 52 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"k9-bench","name":"K9-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","6 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Qwen3-VL-32B-Thinking."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 40.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ogmenrobotics.github.io: 6 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"paprika3d-kaeru-flip","name":"Kaeru Flip (paprika3d)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaet-moonshot","name":"KAET (Kimi Autonomous Execution Tasks)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 83.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-ascii-arena","name":"Kaggle ASCII Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","18 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","7 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 18 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-chess-suite","name":"Kaggle Chess Suite","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-explode-arena","name":"Kaggle Explode Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","6 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-game-arena","name":"Kaggle Game Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","42 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 42 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-landmark-arena","name":"Kaggle Landmark Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","6 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-pelican-arena","name":"Kaggle Pelican Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","26 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","7 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 61 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-sort-viz-arena","name":"Kaggle Sort Viz Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","6 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 13 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"standardized-agent-exams-kaggle","name":"Kaggle Standardized Agent Exams","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","24 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on kaggle.com: 19 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-svg-arena","name":"Kaggle SVG Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","25 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","7 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 25 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-kaggles-task-tuesday-benchmark","name":"Kaggle's Task Tuesday Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kairosqa","name":"KairosQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaleidoscope-cohere","name":"Kaleidoscope","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries) and system cards (1 score)."],["scores.models","pass","7 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Aya Vision 32B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 52.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 7 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"karlbench","name":"KARLBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","10 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on databricks.com: 10 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kat-claw-bench-kuaishou","name":"KAT Claw Bench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for KAT-Coder-Pro V2.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 85.5%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kat-code-bench-kuaishou","name":"KAT Code Bench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for KAT-Coder-V2.5-Dev."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 53.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kazakh-speech-corpus","name":"Kazakh Speech Corpus","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on aclanthology.org: 10 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kazakh-speech-corpus-2","name":"Kazakh Speech Corpus 2","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kbank-mmlu-upstage","name":"KBank-MMLU (Upstage)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (3 scores) and system cards (2 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for Solar Open 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 80.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kbl","name":"KBL","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (4 scores) and system cards (2 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from August 2026, for Solar Pro 4."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 77.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"keep-the-number-you-wrote","name":"Keep the number you wrote","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Sonnet 5.5 (#9)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"keeping-secrets-when-pressed-anthropic","name":"Keeping secrets when pressed in simulated scenarios (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kellybench","name":"KellyBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is -7.90%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kernel-bench-l3","name":"Kernel Bench L3","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kernelbench","name":"KernelBench","summary":{"pass":5,"warn":1,"fail":0,"info":4,"text":"5 of 6 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is MiniMax-M3 (#80)."],["scores.headroom","pass","The best is 28.8%, under 85%, so it still separates models."],["import.importer","warn","The record links a board on scalingintelligence.stanford.edu that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kernelbench-com","name":"KernelBench (kernelbench.com)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kernelbench.com: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kernelbench-verified","name":"KernelBench-Verified","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kernelbench-filtered-metr","name":"KernelBenchFiltered","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kernelbenchinternal-xai","name":"KernelBenchInternal","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","pass","The best is 37.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kernelgen-1p-openai","name":"KernelGen 1P","summary":{"pass":7,"warn":0,"fail":0,"info":3,"text":"All 7 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 76.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kernelquest-turing","name":"KernelQuest","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 79%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.turing.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kernels-task-anthropic","name":"Kernels task (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (11 scores)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kespeech","name":"KeSpeech","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"key-insight-rediscovery-unpublished-ml-task-anthropic","name":"Key insight rediscovery on an unpublished ML task (external testing)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Mythos Preview. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"keyframe-compass","name":"KeyFrame-Compass","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for LTX-2.3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"keystone-eval","name":"Keystone-Eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kgc-safety-lg","name":"KGC-Safety (LG)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from August 2026, for K-EXAONE 2.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 99.8%, past 85% since December 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kgclue","name":"KgCLUE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (354 entries)."],["scores.models","pass","354 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on cluebenchmarks.com: 354 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"khmer-handwritten-dataset-4-2k","name":"Khmer Handwritten Dataset (4.2k)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kiji-pii-detection","name":"Kiji PII Detection","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries)."],["scores.models","pass","41 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on piiarena.com: 41 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reasoning-kili","name":"Kili LLM Reasoning Benchmark","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kilogram-tangrams","name":"KiloGram","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kilometervision","name":"KilometerVision","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on eval.ai: 26 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-chinese-open-ended-eval-moonshot","name":"Kimi Chinese in-house open-ended benchmark","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Kimi K2. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-claw-24-7-bench","name":"Kimi Claw 24/7 Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries) and launch posts (1 score)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from June 2026, for Kimi K2.7 Code."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 52.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kimi.ai: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-code-bench","name":"Kimi Code Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-code-bench-3-long-horizon","name":"Kimi Code Bench 3.0 Long-horizon","summary":{"pass":11,"warn":1,"fail":0,"info":0,"text":"11 of 12 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (1 score)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 66.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kimi.com: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-code-bench-v2","name":"Kimi Code Bench v2","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (3 scores) and system cards (1 score)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3.1 Pro (#17)."],["scores.headroom","pass","The best is 77.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-exploit-development-suite-moonshot","name":"Kimi exploit development suite","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 38.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-hallu-bench","name":"Kimi Hallu Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (1 score)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on kimi.com: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-promptfoo-safety-eval-moonshot","name":"Kimi Promptfoo red-team safety evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Kimi K2. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-vulnerability-discovery-moonshot","name":"Kimi vulnerability discovery (Tier 1)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-webdev-bench-moonshot","name":"Kimi Webdev Bench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 58.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-asr-internal-moonshot","name":"Kimi-ASR Internal Testset","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Kimi-Audio-7B-Instruct. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-asr-internal-moonshot-subset1","name":"Kimi-ASR Internal Testset Subset 1","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","2 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Kimi-Audio-7B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-asr-internal-moonshot-subset2","name":"Kimi-ASR Internal Testset Subset 2","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","2 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Kimi-Audio-7B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-audio-speech-conversation-moonshot","name":"Kimi-Audio speech conversation evaluation","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and system cards (1 score)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Kimi-Audio-7B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kina","name":"KINA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","41 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 53.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 40 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kinayat","name":"Kinayat","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (22 entries)."],["scores.models","pass","8 models we track, plus 14 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash-Lite. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Sonnet 4 (#113)."],["scores.headroom","info","Saturated. The best is 95.3%, past 85% since June 2024."],["import.importer","pass","We read the board on aclanthology.org: 8 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kinetics-400","name":"Kinetics-400","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","1 model we track, plus 15 other entries on its leaderboard."],["scores.recent","info","The newest came 20 months ago, in March 2025, for Qwen2.5-Omni-7B. That’s expected for a test replaced by Kinetics-600."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 0.25%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MTEB: 1 row for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kinetics-600","name":"Kinetics-600","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on SOTA2 Research: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kinetics-700","name":"Kinetics-700","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kinetics-700-2020","name":"Kinetics-700-2020","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","3 models we track, plus 30 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Qwen3-VL-Embedding-2B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MTEB: 8 rows for models we track and 30 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kitab","name":"Kitab","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Phi-4-Reasoning-plus."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 2 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kitti-depth-prediction","name":"KITTI depth prediction","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (58 entries)."],["scores.models","pass","58 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on cvlibs.net: 58 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"klinikebench","name":"KlinikeBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","29 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 29.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on zehui127.github.io: 29 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kmmlu","name":"KMMLU","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (27 entries), launch posts (3 scores) and system cards (5 scores)."],["scores.models","pass","14 models we track, plus 19 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Solar Open 100B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 77%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 19 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kmmlu-hard","name":"KMMLU-Hard","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-4 Turbo (Jan 2024)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 30.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kmmlu-pro","name":"KMMLU-Pro","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (26 entries), launch posts (7 scores) and system cards (6 scores)."],["scores.models","pass","30 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Solar Pro 4."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 79.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 30 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kmmlu-pro-2025","name":"KMMLU-Pro 2025","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kmmlu-pro-2026","name":"KMMLU-Pro 2026","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (26 entries)."],["scores.models","pass","18 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for EXAONE 4.0 32B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 78.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 24 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kmmlu-redux","name":"KMMLU-Redux","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (40 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","32 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for EXAONE 4.0 32B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 81.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 50 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kmmmu","name":"KMMMU","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","13 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for EXAONE 4.5 33B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","pass","The best is 42.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"saxena-knowledge-cutoff","name":"Knowledge Cutoff Benchmark","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on apoorvumang.github.io that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"knowledge-qa-inhouse-minimax","name":"Knowledge Q&A (MiniMax in-house)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for MiniMax-Text-01. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 78.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"knowledgegrapheval-2026","name":"KnowledgeGraphEval 2026","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"knowme-bench","name":"KnowMe-Bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"known-unknown-questions","name":"Known-Unknown Questions","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","2 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in February 2023, for LLaMA-13B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is LLaMA-13B (#234)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"knowu-bench","name":"KnowU-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","6 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-9B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 60.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on zjureal.com: 6 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ko-aime-2024-upstage","name":"Ko-AIME 2024 (Upstage)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 10 months ago, in January 2026, for Solar Open 100B. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 80.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ko-aime-upstage","name":"Ko-AIME 2025 (Upstage)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Solar Open 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 97.7%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ko-arena-hard-v2-upstage","name":"Ko-Arena-Hard v2 (Upstage)","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (2 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Solar Pro 3 (March 2026 update)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 79.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on upstage.ai: 1 row for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ko-gdpval-upstage","name":"Ko-GDPval","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from August 2026, for Solar Pro 4."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 87.3%, past 85% since July 2026."],["import.importer","pass","We read the board on upstage.ai: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ko-ifeval-upstage","name":"Ko-IFEval (Upstage)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1,166 entries) and system cards (1 score)."],["scores.models","pass","22 models we track, plus 1,145 other entries on its leaderboard."],["scores.recent","info","The newest came 10 months ago, in January 2026, for Solar Open 100B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is phi-3-medium 14B (#194)."],["scores.headroom","info","Saturated. The best is 87.5%, past 85% since January 2026."],["import.importer","pass","We read the board on huggingface.co: 24 rows for models we track and 1,145 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ko-longbench-lg","name":"Ko-LongBench (LG)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","3 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for K-EXAONE 2.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 89.6%, past 85% since December 2025."],["import.importer","pass","We read the board on aihub.or.kr: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"koala-test-set","name":"Koala Test Set","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on bair.berkeley.edu: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kobalt","name":"KoBALT","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","15 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for EXAONE 4.5 33B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 61.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 12 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kobest","name":"KoBEST","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kocommongen-v2","name":"KoCommonGEN v2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","1 model we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-8B (#203)."],["scores.headroom","pass","The best is 62.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 11 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kompetencer","name":"Kompetencer","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"komt-bench","name":"KoMT-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 6 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kontext-bench","name":"Kontext Bench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kor-bench","name":"KOR-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (62 entries), launch posts (4 scores) and system cards (2 scores)."],["scores.models","pass","37 models we track, plus 29 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Seed1.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","pass","The best is 77.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kor-bench.github.io: 33 rows for models we track and 29 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kor-phishing-korea-aisi","name":"Kor-Phishing","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kora","name":"KORA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","42 models we track."],["scores.recent","pass","The newest is from October 2026, for Mistral Large 4."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 73.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on korabench.ai: 42 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kora-apps","name":"KORA Apps","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on korabench.ai: 12 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"korean-canonical-legal-benchmark","name":"Korean Canonical Legal Benchmark","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","An importer exists, but it hasn’t read the board yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"korean-contest-prize-reading","name":"Korean Contest Prize Reading","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 Nano (#90)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since March 2026."],["import.importer","pass","We read the board on kaggle.com: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"korean-dental-licensing-2024-2025","name":"Korean Dental Licensing Examination 2024 and 2025","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Claude Opus 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4 (#109)."],["scores.headroom","pass","The best is 81.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on pmc.ncbi.nlm.nih.gov: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"k-nlekmd-2022","name":"Korean National Licensing Examination for Korean Medicine Doctors 2022","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"korean-sarcasm-dataset","name":"Korean Sarcasm Dataset","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (193 entries)."],["scores.models","pass","4 models we track, plus 189 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 4 rows for models we track and 189 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"korgym","name":"KORGym","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","12 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-32B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#104)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on razor233.github.io: 13 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kormedmcqa","name":"KorMedMCQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (31 entries), launch posts (4 scores) and system cards (2 scores)."],["scores.models","pass","11 models we track, plus 24 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Solar Pro 4."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Saturated. The best is 93.2%, past 85% since September 2024."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 24 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kormedmcqa-mixed","name":"KorMedMCQA-Mixed","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (51 entries)."],["scores.models","pass","33 models we track, plus 18 other entries on its leaderboard."],["scores.recent","info","The newest came 11 months ago, in December 2025, for Gemini 3 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Saturated. The best is 97.3%, past 85% since August 2025."],["import.importer","pass","We read the board on kormedmcqa-v.github.io: 33 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kormedmcqa-v","name":"KorMedMCQA-V","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (51 entries)."],["scores.models","pass","33 models we track, plus 18 other entries on its leaderboard."],["scores.recent","info","The newest came 11 months ago, in December 2025, for Gemini 3 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Saturated. The best is 96.9%, past 85% since August 2025."],["import.importer","pass","We read the board on kormedmcqa-v.github.io: 33 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kornat","name":"KorNAT","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","17 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 65.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 17 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"korset-datumo","name":"KorSET","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Gemma 3 12B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-8B (#203)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 20 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kpi-edgar","name":"KPI-EDGAR","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kradlebench","name":"KradleBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kradle.ai: 26 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kradleverse","name":"KradleVerse","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kreta","name":"KRETA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","18 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for EXAONE 4.5 33B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","info","Saturated. The best is 91.9%, past 85% since February 2025."],["import.importer","pass","We read the board on github.com: 17 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kris-bench","name":"KRIS-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries) and system cards (1 score)."],["scores.models","pass","10 models we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Step Image Edit 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on yongliang-wu.github.io: 9 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kryt-benchmark","name":"KRYT Benchmark","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","40 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 32%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 41 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ksaa-2026-speech-diacritization","name":"KSAA 2026 Automatic Diacritization of Speech Dictation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on codabench.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kumo","name":"KUMO","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","5 models we track, plus 10 other entries on its leaderboard."],["scores.recent","info","The newest came 20 months ago, in March 2025, for QwQ-32B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-R1 (#130)."],["scores.headroom","info","Saturated. The best is 86.3%, past 85% since March 2025."],["import.importer","pass","We read the board on huggingface.co: 5 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kuralhub","name":"KuralHub","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aaivu.github.io: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kurdisent","name":"KurdiSent","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kwv-bench-moonshot","name":"KWV Bench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 64.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kyoto-university-commonsense-inference-dataset","name":"Kyoto University Commonsense Inference Dataset","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kyoto-university-wikipedia-annotated-corpus","name":"Kyoto University Wikipedia Annotated Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"l-eval","name":"L-Eval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries) and launch posts (1 score)."],["scores.models","pass","6 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Qwen1.5-72B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-3.5 Turbo (#209)."],["scores.headroom","pass","The best is 78.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on l-eval.github.io: 7 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"l2-bench","name":"L2-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Claude Opus 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on benchmarks.elt.edu.oup.com: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lab-bench","name":"LAB-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries) and system cards (3 scores)."],["scores.models","pass","7 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","pass","The best is 80.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on futurehouse.org: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lab-bench-cloningscenarios","name":"LAB-Bench CloningScenarios","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries), launch posts (1 score) and system cards (12 scores)."],["scores.models","pass","16 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Grok 4.20. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Saturated. The best is 94.2%, past 85% since November 2025."],["import.importer","pass","We read the board on futurehouse.org: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lab-bench-figqa","name":"LAB-Bench FigQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries), launch posts (1 score) and system cards (11 scores)."],["scores.models","pass","15 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","pass","The best is 80.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lab-bench-litqa2","name":"LAB-Bench LitQA2","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in June 2024, for Claude 3.5 Sonnet. That’s expected for a test replaced by LABBench2 LitQA3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","pass","The best is 27%, under 85%, so it still separates models."],["import.importer","pass","We read the board on futurehouse.org: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lab-bench-protocolqa","name":"LAB-Bench ProtocolQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries), launch posts (1 score) and system cards (16 scores)."],["scores.models","pass","20 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Saturated. The best is 90.7%, past 85% since November 2025."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lab-bench-seqqa","name":"LAB-Bench SeqQA","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (12 scores)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Saturated. The best is 98.2%, past 85% since November 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lab-contracts","name":"LAB: Contracts","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lab-diligence","name":"LAB: Diligence","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lab-law-firm-knowledge","name":"LAB: Law Firm Knowledge","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lab-uscl","name":"LAB: US Case Law","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"labbench2","name":"LABBench2","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 88.8%, past 85% since September 2026."],["import.importer","pass","We read the board on advances.edisonscientific.com: 10 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"labbench2-cloning","name":"LABBench2 CloningQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","pass","The best is 42.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on advances.edisonscientific.com: 27 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"labbench2-figqa2","name":"LABBench2 FigQA2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","pass","The best is 42.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on advances.edisonscientific.com: 10 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"labbench2-figqa2-img","name":"LABBench2 FigQA2 (image)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","pass","The best is 66.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on advances.edisonscientific.com: 10 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"labbench2-figqa2-pdf","name":"LABBench2 FigQA2 (PDF)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","pass","The best is 64.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on advances.edisonscientific.com: 10 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"labbench2-litqa3","name":"LABBench2 LitQA3","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","info","Saturated. The best is 85.1%, past 85% since December 2025."],["import.importer","pass","We read the board on advances.edisonscientific.com: 10 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"labbench2-patentqa","name":"LABBench2 PatentQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","info","Saturated. The best is 90.9%, past 85% since December 2025."],["import.importer","pass","We read the board on advances.edisonscientific.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"labbench2-protocolqa2","name":"LABBench2 ProtocolQA2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","pass","The best is 61.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on advances.edisonscientific.com: 10 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"labbench2-seqqa2","name":"LABBench2 SeqQA2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","pass","The best is 52.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on advances.edisonscientific.com: 26 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"labbench2-sourcequality","name":"LABBench2 SourceQuality","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","info","Saturated. The best is 90%, past 85% since November 2025."],["import.importer","pass","We read the board on advances.edisonscientific.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"labbench2-tableqa2-img","name":"LABBench2 TableQA2 (image)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","info","Saturated. The best is 95%, past 85% since November 2025."],["import.importer","pass","We read the board on advances.edisonscientific.com: 10 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"labbench2-tableqa2-pdf","name":"LABBench2 TableQA2 (PDF)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","info","Saturated. The best is 88%, past 85% since November 2025."],["import.importer","pass","We read the board on advances.edisonscientific.com: 10 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"labbench2-trialqa","name":"LABBench2 TrialQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","info","Saturated. The best is 93.3%, past 85% since December 2025."],["import.importer","pass","We read the board on advances.edisonscientific.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-search-labelbox","name":"Labelbox Agentic Search","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labelbox.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"complex-reasoning-labelbox","name":"Labelbox Complex Reasoning","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","11 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 89.6%, past 85% since April 2025."],["import.importer","pass","We read the board on labelbox.com: 11 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deep-research-agents-labelbox","name":"Labelbox Deep Research Agents","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 77.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labelbox.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"image-generation-labelbox","name":"Labelbox Image Generation (original)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"image-generation-labelbox-2","name":"Labelbox Image Generation v2","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for GPT-4.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4.1 (#138)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on labelbox.com: 5 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multimodal-reasoning-labelbox","name":"Labelbox Multimodal Reasoning","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on labelbox.com: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speech-generation-labelbox","name":"Labelbox Speech Generation (original)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speech-generation-labelbox-2","name":"Labelbox Speech Generation v2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on labelbox.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video-generation-labelbox","name":"Labelbox Video Generation (original)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video-generation-labelbox-2","name":"Labelbox Video Generation v2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","2 models we track, plus 5 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on labelbox.com: 2 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lad2000","name":"LAD2000","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","An importer exists, but it hasn’t read the board yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ladder-robotics","name":"LADDER","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","2 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 63.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ladderrobotics.ai: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lahjoita-puhetta-asr","name":"Lahjoita puhetta ASR test","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lakera-b3-backbone-breaker-benchmark","name":"Lakera B3 (Backbone Breaker Benchmark)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries) and launch posts (1 score)."],["scores.models","pass","36 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Mistral Large 4 (Le Chonk, open weights)."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Saturated. The best is 97.1%, past 85% since October 2025."],["import.importer","pass","We read the board on b3.lakera.ai: 52 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lakh-midi-masd-style","name":"Lakh MIDI MASD Style Classification","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lakh-midi-top-magd-genre","name":"Lakh MIDI Top-MAGD Genre Classification","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on hdl.handle.net: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lakh-crore-and-carpet-area","name":"Lakh, Crore and Carpet Area","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Haiku 5.5 (#20)."],["scores.headroom","info","Saturated. The best is 97%, past 85% since February 2026."],["import.importer","pass","We read the board on kaggle.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lambada","name":"LAMBADA","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and claims Epoch AI collected (26 models)."],["scores.models","pass","27 models we track."],["scores.recent","info","The newest came 3 years ago, in November 2023, for Qwen-1_8B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is PaLM 2-L (#198)."],["scores.headroom","info","Saturated. The best is 87.2%, past 85% since October 2021."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lambada-multilingual-stable-lm-2","name":"LAMBADA Multilingual (Stable LM 2)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","3 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Gemma 2B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Phi-2 (#225)."],["scores.headroom","pass","The best is 39.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lambada-openai-multilingual","name":"LAMBADA OpenAI Multilingual","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lamina-product-bench","name":"Lamina Product Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"laminabench-v1","name":"LaminaBench v1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-landing-page-arena","name":"Landing Page Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","6 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 14 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"langchain-benchmarks","name":"LangChain Benchmarks","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"language-barrier-benchmark","name":"Language Barrier Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 0%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"language-confusion-benchmark","name":"Language Confusion Benchmark","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Cohere Command A. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 93%, past 85% since April 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"language-table","name":"Language-Table","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"large-language-model-football-league","name":"Large Language Model Football League","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on modelfootball.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"large-movie-review-dataset","name":"Large Movie Review Dataset","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 10 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"large-vocabulary-video-instance-segmentation","name":"Large-Vocabulary Video Instance Segmentation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"larybench","name":"LARYBench","summary":{"pass":1,"warn":3,"fail":0,"info":6,"text":"1 of 4 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lscbench","name":"Last Secure Code Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (1 score)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from November 2026, for Qwen4-Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 31.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on last-secure-code-benchmark.github.io: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"last-translation-benchmark","name":"Last Translation Benchmark","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (52 entries)."],["scores.models","pass","37 models we track, plus 15 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Mistral Large 4."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 45.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on last-translation-benchmark.vilda.net: 37 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"latchbio-capabilities-v0","name":"LatchBio Capabilities V0","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (46 entries)."],["scores.models","pass","17 models we track, plus 29 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 51.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on LatchBio: 46 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"latchbio-capabilities-v1-0","name":"LatchBio Capabilities v1.0","summary":{"pass":10,"warn":1,"fail":1,"info":0,"text":"10 of 12 checks pass, 1 warning, 1 problem"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","fail","Unverified. We haven’t checked it against a primary source yet."],["scores.sources","pass","Scores from the official leaderboard (27 entries) and system cards (1 score)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 58%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 42 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"latin-synonym-selection","name":"Latin Synonym Selection Benchmark","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on ceur-ws.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lawbench","name":"LawBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (51 entries)."],["scores.models","pass","11 models we track, plus 40 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for Qwen-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-3.5 Turbo (#209)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 22 rows for models we track and 40 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"laymaker-mtsamples","name":"LayMaker MTSamples Simplification","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lazy-investigation-anthropic","name":"Lazy investigation (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lazy-investigation-misleading-codebases-anthropic","name":"Lazy investigation on misleading codebases (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lbm-eval","name":"LBM Eval","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Foundry-Qwen3VLA-2.1B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 72.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lbpp","name":"LBPP (Less Basic Python Problems)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (28 entries) and system cards (2 scores)."],["scores.models","pass","22 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for DiffusionGemma 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","pass","The best is 81%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 20 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lcbench-opencompass","name":"LCBench v0.1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"leandojo-benchmark","name":"LeanDojo Benchmark","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on leandojo.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"leanlean","name":"LeanLean","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 50.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on matharena.ai: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"learn2play-bench","name":"Learn2Play Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on liushiliushi.github.io: 31 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lech-mazur-creative-writing","name":"Lech Mazur Creative Writing","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (62 entries)."],["scores.models","pass","55 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on raw.githubusercontent.com: 60 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ledgerbench-100","name":"LedgerBench-100","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on blobfish.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"leeds-sports-pose-extended","name":"Leeds Sports Pose Extended","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"leetcode-easy-openai","name":"Leetcode (easy)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 3 years ago, in March 2023, for GPT-4. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-4 (#181)."],["scores.headroom","pass","The best is 75.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"leetcode-hard-openai","name":"Leetcode (hard)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 3 years ago, in March 2023, for GPT-4. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-4 (#181)."],["scores.headroom","pass","The best is 6.70%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"leetcode-medium-openai","name":"Leetcode (medium)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 3 years ago, in March 2023, for GPT-4. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-4 (#181)."],["scores.headroom","pass","The best is 26.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"leetcode-weekly-contest","name":"LeetCode Weekly Contest","summary":{"pass":9,"warn":2,"fail":0,"info":1,"text":"9 of 11 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries) and system cards (1 score)."],["scores.models","pass","2 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for DeepSeek LLM 67B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 17.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legacy-bench","name":"Legacy-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 60%, under 85%, so it still separates models."],["import.importer","pass","We read the board on factory.com: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legacyshop","name":"LegacyShop","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legal-consent-sweep","name":"Legal Consent Sweep","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on hub.harborframework.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"firecrawl-legal-retrieval","name":"Legal Information Retrieval Benchmark","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on firecrawl.dev: 4 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legal-opinion-sentiment-classification","name":"Legal Opinion Sentiment Classification","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legal-research-bench","name":"Legal Research Bench","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (99 entries), launch posts (3 scores) and system cards (1 score)."],["scores.models","pass","97 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 56.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 98 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legal-world-sample","name":"Legal World sample","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legalbench","name":"LegalBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (139 entries)."],["scores.models","pass","138 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GLM Legal."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","info","Saturated. The best is 88.6%, past 85% since August 2025."],["import.importer","pass","We read the board on Vals AI: 150 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legalbench-pt","name":"LegalBench.PT","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","info","The newest came 20 months ago, in March 2025, for GPT-4o. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","info","Saturated. The best is 85.4%, past 85% since June 2024."],["import.importer","pass","We read the board on github.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legalcitebench","name":"LegalCiteBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","14 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Claude Haiku 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 4.5 (#81)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 14 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legalquad","name":"LegalQuAD","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","4 models we track, plus 38 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on VAGO Solutions: 4 rows for models we track and 38 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legalreasonbench","name":"LegalReasonBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","5 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 50.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on micro1.ai: 11 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legalrewardbench","name":"LegalRewardBench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (38 entries)."],["scores.models","pass","29 models we track, plus 9 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for GPT-5.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 86.8%, past 85% since November 2025."],["import.importer","pass","We read the board on arxiv.org: 29 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legalsupport","name":"LegalSupport","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lego-puzzles","name":"LEGO-Puzzles","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (29 entries)."],["scores.models","pass","16 models we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Qwen3-VL-32B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 72%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 16 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"legora-bar","name":"Legora BAR","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on legora.com: 17 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"leipzig-benchmark","name":"Leipzig Benchmark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 91%, past 85% since July 2026."],["import.importer","pass","We read the board on math.science-bench.ai: 20 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lengthctrl-qwen","name":"LengthCtrl (Qwen)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Qwen3-32B. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 73.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lengthctrl-tencent","name":"LengthCtrl (Tencent)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Hunyuan-A13B-Instruct. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 55.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"letta-leaderboard","name":"Letta Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on docs.letta.com: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"levelbench-100-4-1","name":"LevelBench-100 v4.1","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","22 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 81%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 38 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"levelbench-100-5","name":"LevelBench-100 v5.0","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","19 models we track, plus 23 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 95.2%, past 85% since May 2026."],["import.importer","pass","We read the board on levelcfo.com: 61 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"levir-cc","name":"LEVIR-CC","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on levir.buaa.edu.cn: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"levir-cdc","name":"LEVIR-CDC","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lewitt-bench","name":"LeWitt-Bench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","warn","The record links a board on designarena.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lexam","name":"LEXam","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (35 entries)."],["scores.models","pass","31 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 70.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on lexam-benchmark.github.io: 32 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lexam-hard","name":"LEXam-hard","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","15 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Apertus 1.5 70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 40.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 15 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lexgenius","name":"LexGenius","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-8B (#154)."],["scores.headroom","pass","The best is 59.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 16 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lexglue","name":"LexGLUE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lexpilot","name":"LexPilot","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","8 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 72.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lextreme","name":"LEXTREME","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lgeb","name":"LGEB","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lhaw","name":"LHAW","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 68.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lhrs-bench","name":"LHRS-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","3 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Claude 3 Opus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 39.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"libero","name":"LIBERO","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (6 scores) and system cards (1 score)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for MolmoAct 2. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 97.2%, past 85% since February 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"libero-max","name":"LIBERO-MAX","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","4 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for MolmoAct 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 66.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"libero-para","name":"LIBERO-Para","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for π0.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 71.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cau-hai-lab.github.io: 1 row for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"libero-plus","name":"LIBERO-Plus","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","3 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for OpenVLA-OFT."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 69.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"libero-pro","name":"LIBERO-PRO","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for π0.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 53%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"libra-russian-long-context","name":"LIBRA (Russian Long Context)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","9 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-0.8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5-35B-A3B (#103)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 9 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"librarydesignbench","name":"LibraryDesignBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ldbench.com: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"libricount","name":"LibriCount","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (52 entries)."],["scores.models","pass","1 model we track, plus 51 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for Whisper large-v3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 57.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MTEB: 1 row for models we track and 51 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"librispeech","name":"LibriSpeech","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (8 scores) and system cards (2 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Seed2.0 Lite (0428)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"life-bench","name":"Life-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Gemma 3 12B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 56%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lifechoice","name":"LifeChoice","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for o1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 65.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lifesci-diamond","name":"LifeSci Diamond","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","18 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 20%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 21 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lifescibench","name":"LifeSciBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (4 scores) and system cards (1 score)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-Rosalind-6."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 61.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openai.com: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lifeval","name":"LifeVal","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 86.1%, past 85% since November 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lightgenbench","name":"LightGenBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lights-out-3x3","name":"Lights Out 3×3","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"limit","name":"LIMIT","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"limmits-2024","name":"LIMMITS 2024","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on sites.google.com: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"limmits-2025","name":"LIMMITS 2025","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on sites.google.com: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ladr-axler","name":"Linear Algebra Done Right (Axler Exercises)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lineex","name":"LineEX","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lingoly","name":"LingOly","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 46.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lingoly-too","name":"LingOly-TOO","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for DeepSeek-V3.1-Terminus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 46.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 15 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lingoqa","name":"LingoQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries), launch posts (8 scores) and system cards (2 scores)."],["scores.models","pass","11 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 84.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"linguasafe","name":"LinguaSafe","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","7 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Apertus 70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3 (#141)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"linguini","name":"Linguini","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries) and system cards (1 score)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Glimmer."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 24.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 91 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fab-linguistic-knowledge","name":"Linguistic Knowledge Benchmark (Fab AI)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (115 entries)."],["scores.models","pass","109 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 97.8%, past 85% since April 2025."],["import.importer","pass","We read the board on fab-ai.org: 115 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"linuxarena","name":"LinuxArena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries) and system cards (6 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on linuxarena.ai: 3 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lisanbench","name":"LisanBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (137 entries)."],["scores.models","pass","133 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on lisanbench.com: 207 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"litbench","name":"LitBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","An importer exists, but it hasn’t read the board yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"litqa-v0-futurehouse","name":"LitQA v0","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","2 models we track, plus 8 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in July 2023, for Claude 2. That’s expected for a test replaced by LAB-Bench LitQA2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 (#186)."],["scores.headroom","pass","The best is 40.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"litsearch","name":"LitSearch","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"live-gaming-benchmark","name":"Live Gaming Benchmark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","1 model we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for LiveCC-7B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 28.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livebench","name":"LiveBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries), launch posts (14 scores) and system cards (22 scores)."],["scores.models","pass","63 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 87.4%, past 85% since November 2026."],["import.importer","pass","We read the board on livebench.ai: 36 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"liveclawbench","name":"LiveClawBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries) and launch posts (2 scores)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 76%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mosi-ai.github.io: 17 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livecodebench","name":"LiveCodeBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries), launch posts (38 scores) and system cards (26 scores)."],["scores.models","pass","75 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.7-Max (#49)."],["scores.headroom","info","Saturated. The best is 93.5%, past 85% since April 2025."],["import.importer","pass","We read the board on livecodebench.github.io: 27 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livecodebench-pro","name":"LiveCodeBench Pro","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries), launch posts (8 scores) and system cards (5 scores)."],["scores.models","pass","18 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 15 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livecodebench-v5","name":"LiveCodeBench v5","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries), launch posts (35 scores) and system cards (32 scores)."],["scores.models","pass","65 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Nemotron 3 Nano Omni 30B-A3B. That’s expected for a test replaced by LiveCodeBench v6."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 81.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on livecodebench.github.io: 27 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livecodebench-v6","name":"LiveCodeBench v6","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries), launch posts (75 scores) and system cards (53 scores)."],["scores.models","pass","111 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 27B (#59)."],["scores.headroom","info","Saturated. The best is 93.5%, past 85% since August 2025."],["import.importer","pass","We read the board on livecodebench.github.io: 27 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livedrbench-microsoft","name":"LiveDRBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for OpenAI deep research."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 55.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livemace","name":"LiveMACE","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livemath-google","name":"LiveMath","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livemathbench","name":"LiveMathBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","8 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for o3-mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3-mini (#124)."],["scores.headroom","pass","The best is 76.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livemathematicianbench","name":"LiveMathematicianBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries) and system cards (1 score)."],["scores.models","pass","18 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 45.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on livemathematicianbench.github.io: 17 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livenewsbench","name":"LiveNewsBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 11 months ago, in December 2025, for GPT-5.2. That’s expected for a test replaced by LiveNewsBench Release 2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 21.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livenewsbench-2","name":"LiveNewsBench Release 2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Kimi K2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 84.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on livenewsbench.com: 41 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"liveocrvqa","name":"LiveOCRVQA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","18 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on flageval-baai.github.io: 18 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"liveresearchbench","name":"LiveResearchBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","8 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on livedeepresearch.github.io: 10 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livesearch","name":"LiveSearch","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for Step 5 Preview."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 76.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livesecbench","name":"LiveSecBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (43 entries)."],["scores.models","pass","34 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on livesecbench.intokentech.cn: 34 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livesports-3k","name":"LiveSports-3K","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (3 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 78.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livesqlbench","name":"LiveSQLBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (43 entries) and launch posts (1 score)."],["scores.models","pass","36 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 40.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on livesqlbench.ai: 44 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"livetradebench","name":"LiveTradeBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","18 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 12.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 18 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llama-2-helpfulness-human-evaluation-meta","name":"Llama 2-Chat helpfulness human evaluation (Meta)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 3 years ago, in July 2023, for Llama 2-13B. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llama-2-safety-human-evaluation-meta","name":"Llama 2-Chat safety human evaluation (Meta)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 3 years ago, in July 2023, for Llama 2-13B. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llama-human-evaluation-meta","name":"Llama 3 pairwise human evaluation (Meta)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in November 2024, for Llama 3.1-405B. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llama-internal-safety-benchmarks-meta","name":"Llama internal safety benchmarks (Meta)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in November 2024, for Llama 3.1-405B. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"verbatim-memorization-meta","name":"Llama verbatim memorization probe (Meta)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in November 2024, for Llama 3.1-405B. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llamatouch","name":"LlamaTouch","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llava-bench-in-the-wild","name":"LLaVA-Bench (In-the-Wild)","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-3d-worlds-arena","name":"LLM 3D Worlds Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on 3d-arena-llm-3d-worlds-arena.hf.space: 24 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-agronomy-benchmark-fbn","name":"LLM Agronomy Benchmark (Farmers Business Network edition)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (107 entries)."],["scores.models","pass","94 models we track, plus 13 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 94.3%, past 85% since April 2025."],["import.importer","pass","We read the board on github.com: 97 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-agronomy-benchmark-june-2024","name":"LLM Agronomy Benchmark (June 2024)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","8 models we track, plus 7 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in June 2024, for Qwen2-72B. That’s expected for a test replaced by LLM Agronomy Benchmark (Farmers Business Network edition)."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Saturated. The best is 89.6%, past 85% since February 2024."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-agronomy-benchmark-original","name":"LLM Agronomy Benchmark (original)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","3 models we track, plus 5 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in November 2023, for GPT-3.5 Turbo. That’s expected for a test replaced by LLM Agronomy Benchmark (June 2024)."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","pass","The best is 83.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-agronomy-benchmark-image-identification-four-choice","name":"LLM Agronomy Benchmark Image Identification (four choices)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (38 entries)."],["scores.models","pass","37 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 Pro (#19)."],["scores.headroom","info","Saturated. The best is 90%, past 85% since June 2025."],["import.importer","pass","We read the board on github.com: 38 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"idiom-dk-danske-idiomer","name":"LLM Benchmark: Danske Idiomer","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","18 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 11 months ago, in December 2025, for GPT-5.2. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Saturated. The best is 97.5%, past 85% since July 2024."],["import.importer","pass","We read the board on idiom.dk: 18 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-chess","name":"LLM Chess","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (133 entries)."],["scores.models","pass","123 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Muse Ember."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on maxim-saplin.github.io: 178 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-chess-arena-louis-guichard","name":"LLM Chess Arena (Louis Guichard)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (47 entries)."],["scores.models","pass","47 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Terra (#18)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on chess.louisguichard.fr: 47 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-colosseum","name":"LLM Colosseum","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","3 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for GPT-4o mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o mini (#178)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 6 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lech-mazur-confabulations","name":"LLM Confabulation Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (60 entries)."],["scores.models","pass","59 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 65 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"debate-lech-mazur","name":"LLM Debate Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (66 entries)."],["scores.models","pass","58 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 60 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deception-lech-mazur","name":"LLM Deceptiveness and Gullibility","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"divergent-lech-mazur","name":"LLM Divergent Thinking Creativity Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","18 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 18 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-economist","name":"LLM Economist","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","An importer exists, but it hasn’t read the board yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-fantasy-football-arena-classic","name":"LLM Fantasy Football Arena Classic","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arena.wildcardlabs.tech: 24 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-fantasy-football-arena-guillotine","name":"LLM Fantasy Football Arena Guillotine","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","6 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arena.wildcardlabs.tech: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-game-benchmark","name":"LLM Game Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","3 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for Claude 3.5 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on research-outcome.github.io: 3 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-games-ramp","name":"LLM Games","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on labs.ramp.com: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"persuasion-lech-mazur","name":"LLM Persuasion Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","11 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for MiniMax-M2.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 11 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-physics-plotting-benchmark","name":"LLM Physics Plotting Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","17 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Saturated. The best is 90.6%, past 85% since August 2025."],["import.importer","pass","We read the board on gordonwatts.github.io: 44 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"position-bias-lech-mazur","name":"LLM Position Bias Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","31 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 36 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"public-goods-game-lech-mazur","name":"LLM Public Goods Game","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","19 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for DeepSeek-R1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 45.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 20 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"translation-lech-mazur","name":"LLM Round-Trip Translation Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Kimi K2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-soccerarena-bundesliga-2026-27","name":"LLM SoccerArena Bundesliga 2026/27","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on llm-soccerarena.com: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-soccerarena-la-liga-2026-27","name":"LLM SoccerArena La Liga 2026/27","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on llm-soccerarena.com: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-soccerarena-premier-league-2026-27","name":"LLM SoccerArena Premier League 2026/27","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on llm-soccerarena.com: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-soccerarena-world-cup-2026","name":"LLM SoccerArena World Cup 2026","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on llm-soccerarena.com: 96 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sycophancy-lech-mazur","name":"LLM Sycophancy Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","20 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Qwen3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 20 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"thematic-generalization-v2","name":"LLM Thematic Generalization Benchmark v2","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","19 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Claude Opus 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 25 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-training-puzzles","name":"LLM Training Puzzles","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-training-task-anthropic","name":"LLM training task (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (11 scores)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-uncertainty-bench","name":"LLM Uncertainty Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","14 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for Qwen-1_8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-70B (#210)."],["scores.headroom","pass","The best is 81.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 14 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-writer-negative-style-prime","name":"LLM Writer Negative Style","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for GPT-4.1 mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4.1 mini (#145)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on app.primeintellect.ai: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-aggrefact","name":"LLM-AggreFact","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (47 entries)."],["scores.models","pass","10 models we track, plus 37 other entries on its leaderboard."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for Llama 3.3 70B Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 75.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on llm-aggrefact.github.io: 10 rows for models we track and 37 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-cae-benchmark","name":"LLM-CAE-Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","5 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in April 2024, for Llama 3-70B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3-70B (#182)."],["scores.headroom","info","Saturated. The best is 90%, past 85% since April 2024."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-coordination","name":"LLM-Coordination","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in December 2023, for Mixtral 8x7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Mixtral 8x7B (#199)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-ioaa","name":"LLM-IOAA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 86.3%, past 85% since August 2025."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-jp-eval","name":"llm-jp-eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-nerre-materials","name":"LLM-NERRE materials extraction","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-perf-leaderboard","name":"LLM-Perf Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (46 entries)."],["scores.models","pass","21 models we track, plus 25 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for Qwen2-0.5B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is phi-3-mini 3.8B (#204)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 1,299 rows for models we track and 25 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-srbench","name":"LLM-SRBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-8B (#203)."],["scores.headroom","pass","The best is 15.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-wikirace","name":"LLM-WikiRace","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries)."],["scores.models","pass","22 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 23%, under 85%, so it still separates models."],["import.importer","pass","We read the board on llmwikirace.github.io: 22 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llmbar","name":"LLMBar","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llmboxing","name":"LLMBoxing","summary":{"pass":6,"warn":2,"fail":0,"info":4,"text":"6 of 8 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in October 2023, for Mistral 7B v0.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-70B (#210)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on web.archive.org: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llmeval-med","name":"LLMEval-Med","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llmzsz","name":"LLMzSzŁ","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (98 entries)."],["scores.models","pass","32 models we track, plus 66 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 69.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 33 rows for models we track and 66 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lm-syneval","name":"LM-SynEval","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lmact","name":"LMAct","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lmarcade-afterquery","name":"LMArcade","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (160 entries)."],["scores.models","pass","38 models we track, plus 122 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on lmarcade.com: 193 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lmarena-elo","name":"LMArena (Elo)","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (390 entries), launch posts (4 scores) and system cards (10 scores)."],["scores.models","pass","346 models we track, plus 45 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arena.ai: 381 rows for models we track and 45 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 13 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lmarena-image-edit","name":"LMArena Image Edit Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (60 entries)."],["scores.models","pass","58 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for BAGEL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arena.ai: 62 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lmarena-image-to-video","name":"LMArena Image-to-Video Arena","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (47 entries)."],["scores.models","pass","45 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Dreamina Seedance 2.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arena.ai: 48 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lmarena-search-arena","name":"LMArena Search Arena","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries) and launch posts (2 scores)."],["scores.models","pass","39 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arena.ai: 40 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lmarena-text-to-image","name":"LMArena Text-to-Image Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (81 entries)."],["scores.models","pass","79 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for BAGEL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arena.ai: 84 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lmarena-text-to-video","name":"LMArena Text-to-Video Arena","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (48 entries)."],["scores.models","pass","46 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Dreamina Seedance 2.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arena.ai: 48 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lmarena-video-edit","name":"LMArena Video Edit Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Dreamina Seedance 2.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arena.ai: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lmbuild","name":"LMBuild","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","18 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 66.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on lumos-jiateng.github.io: 18 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lmca","name":"LMCA","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (156 entries)."],["scores.models","pass","149 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 70.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on conceptualreasoning.ai: 213 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lmentry","name":"LMentry","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","9 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 4 years ago, in March 2022, for davinci-002."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 66.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 9 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lmgame-bench","name":"lmgame-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","23 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 24 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lmsys-chatbot-arena-human-preference-predictions","name":"LMSYS Chatbot Arena Human Preference Predictions","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1,688 entries)."],["scores.models","pass","1,688 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 1,688 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"loca-bench","name":"LOCA-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries) and launch posts (2 scores)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 60.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on loca-bench.github.io: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aox-local-llm-arkanoid-test","name":"Local LLM Arkanoid Test","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","10 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aox.cz: 11 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"aox-local-llm-tetris-test","name":"Local LLM Tetris Test","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aox.cz: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"local-search-bench-meituan","name":"LocalSearchBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 35.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on localsearchbench.github.io: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"locomo","name":"LoCoMo","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on snap-research.github.io: 6 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"loft","name":"LOFT","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (6 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Saturated. The best is 87%, past 85% since July 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"logicval","name":"LogicVal","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"logicvista","name":"LogicVista","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (7 scores) and system cards (12 scores)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from August 2026, for LFM2.5-VL-3B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","info","Saturated. The best is 91.9%, past 85% since August 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"logiqa","name":"LogiQA","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"loki","name":"LOKI","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries)."],["scores.models","pass","15 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for Qwen2-VL-72B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 60.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 15 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"long-code-arena","name":"Long Code Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","5 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for o1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 5 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"long-context-inhouse-minimax","name":"Long Context (MiniMax in-house)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for MiniMax-Text-01. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 93.8%, past 85% since January 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"long-context-navigator","name":"Long Context Navigator","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 1 entry at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"long-range-arena","name":"Long Range Arena","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 13 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"long-context-grounding-unfactual-rate-google","name":"Long-context grounding, sentence unfactual rate","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"long-document-qa-kensho","name":"Long-document QA (Kensho)","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on benchmarks.kensho.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"long-form-biorisk-questions-openai","name":"Long-form Biological Risk Questions","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for GPT-5.1-Codex-Max. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-4.5 (#136)."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"long-form-virology-tasks-anthropic","name":"Long-form virology tasks","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 69.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"long-horizon-freight-auditbench-challenge","name":"Long-Horizon Freight AuditBench Challenge","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 10%, under 85%, so it still separates models."],["import.importer","pass","We read the board on loop.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"long-horizon-freight-auditbench-production","name":"Long-Horizon Freight AuditBench Production","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 16.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on loop.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"long-horizon-jailbreaks-xai","name":"Long-horizon jailbreaks (xAI)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"long-horizon-medical-agent-vet-clinic-ops-2","name":"Long-Horizon Medical Agent Benchmark, Veterinary Clinic Ops v2","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since July 2026."],["import.importer","pass","We read the board on app.primeintellect.ai: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"long-horizon-terminal-bench-lhtb","name":"Long-Horizon-Terminal-Bench (LHTB)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","25 models we track."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 60.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 26 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"long-horizon-terminal-bench-2-0","name":"Long-Horizon-Terminal-Bench 2.0","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 69.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 15 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"long-term-video-object-segmentation-v2","name":"Long-term Video Object Segmentation v2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","19 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 19 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longaudiospan","name":"LongAudioSpan","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","10 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 82.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 11 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longav-compass","name":"LongAV-Compass","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longbanana","name":"LongBanana","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","1 model we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Flash Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on klingairesearch.github.io: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longbench","name":"LongBench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for EXAONE 4.0 32B. That’s expected for a test replaced by LongBench v2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 81.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longbench-chinese","name":"LongBench Chinese","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longbench-english","name":"LongBench English","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries) and system cards (1 score)."],["scores.models","pass","1 model we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for EXAONE 4.0 32B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 48.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longbench-pro","name":"LongBench Pro","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (45 entries)."],["scores.models","pass","36 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Ministral 3 3B Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 73.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 63 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longbench-v2","name":"LongBench v2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries), launch posts (28 scores) and system cards (19 scores)."],["scores.models","pass","67 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 70.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on longbench2.github.io: 55 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longbench-chat","name":"LongBench-Chat","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (14 entries), launch posts (1 score) and system cards (6 scores)."],["scores.models","pass","9 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Qwen2.5-14B-Instruct-1M."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on raw.githubusercontent.com: 3 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longcat-safety-meituan","name":"LongCat safety evaluation (Meituan)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for LongCat-Flash-Thinking. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 95.7%, past 85% since August 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longcodebench","name":"LongCodeBench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Nova 2.0 Pro Preview."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 84%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longcot","name":"LongCoT","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 25.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on longcot.ai: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longdocurl","name":"LongDocURL","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries) and system cards (1 score)."],["scores.models","pass","6 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Seed2.0 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 74.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longextract-bench","name":"LongExtractionBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","3 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 70.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on micro1.ai: 3 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longfact","name":"LongFact","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (5 scores) and system cards (1 score)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from June 2026, for MAI-Thinking-1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longfact-concepts","name":"LongFact-Concepts","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Seed2.0 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longfact-objects","name":"LongFact-Objects","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries) and system cards (4 scores)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Seed2.0 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on OpenAI: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"eqbench-longform-writing","name":"Longform Creative Writing (EQ-Bench)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (139 entries)."],["scores.models","pass","120 models we track, plus 19 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eqbench.com: 124 rows for models we track and 19 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longform-writing","name":"Longform Writing","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Kimi K2 Thinking. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K2 Thinking (#94)."],["scores.headroom","pass","The best is 73.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longhealth","name":"LongHealth","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","2 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in December 2023, for Mistral-7B-Instruct-v0.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-3.5 Turbo (#201)."],["scores.headroom","pass","The best is 51.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longjud-bench","name":"LongJud-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longleader","name":"LongLeader","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longmemeval","name":"LongMemEval","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.1-70B (#177)."],["scores.headroom","info","Saturated. The best is 92.4%, past 85% since March 2025."],["import.importer","pass","We read the board on arxiv.org: 162 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longmemeval-v2","name":"LongMemEval-V2","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-9B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5-9B (#128)."],["scores.headroom","pass","The best is 1.30%, under 85%, so it still separates models."],["import.importer","pass","We read the board on xiaowu0162.github.io: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longproc","name":"LongProc","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","14 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in October 2024, for Claude 3.5 Sonnet (October 2024)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (October 2024) (#150)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on princeton-pli.github.io: 14 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longpuzzlebench","name":"LongPuzzleBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","An importer exists, but it hasn’t read the board yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longshotbench","name":"LongShOTBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (106 entries)."],["scores.models","pass","67 models we track, plus 39 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen3.6 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 64.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on longshot.cvmbzuai.com: 67 rows for models we track and 39 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longtail-swe","name":"Longtail-SWE","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 81.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longtext-bench","name":"LongText-Bench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for GLM-Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"longvideobench","name":"LongVideoBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (35 entries), launch posts (8 scores) and system cards (10 scores)."],["scores.models","pass","35 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K2.5 (#76)."],["scores.headroom","pass","The best is 80.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on longvideobench.github.io: 23 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"looparena","name":"LoopArena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 36.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"loopsbench","name":"LoopsBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 25%, under 85%, so it still separates models."],["import.importer","pass","We read the board on loopsbench.ai: 14 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"loss-run-extraction-bevaya","name":"Loss Run Extraction (Bevaya)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","12 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 85.3%, past 85% since June 2026."],["import.importer","pass","We read the board on bevaya.ai: 18 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lotte","name":"LoTTE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on MTEB: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"loveda","name":"LoveDA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"loveu-tgve-2023","name":"LOVEU-TGVE-2023","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lp-tts-bench","name":"LP-TTS-Bench","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lpfqa","name":"LPFQA","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Seed1.8."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 52.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lsat","name":"LSAT","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (7 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lsmdc","name":"LSMDC","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 3 years ago, in March 2023, for GPT-4. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","pass","The best is 45.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lt-history","name":"LT-History","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","6 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Llama 3.2 3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 82%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 6 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ltzglue","name":"ltzGLUE","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","4 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5 mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 mini (#92)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ludobench-backgammon","name":"Ludobench Backgammon","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on ludobench.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ludobench-hokm","name":"Ludobench Hokm","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since October 2025."],["import.importer","pass","We read the board on ludobench.com: 8 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lv-eval","name":"LV-Eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for Qwen2.5-14B-Instruct-1M. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 49.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lvbench","name":"LVBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries), launch posts (30 scores) and system cards (19 scores)."],["scores.models","pass","40 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 91.7%, past 85% since August 2026."],["import.importer","pass","We read the board on lvbench.github.io: 10 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lvbench-with-memory","name":"LVBench (with memory)","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","info","Saturated. The best is 85.6%, past 85% since August 2026."],["import.importer","warn","The record links a board on lvbench.github.io that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lvis","name":"LVIS","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lvis-mg-bytedance","name":"LVIS-MG","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Seed1.5-VL. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 72.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lvomnibench","name":"LVOmniBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","7 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 65.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kd-tao.github.io: 7 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lvsum","name":"LVSum","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lying-about-identity-anthropic","name":"Lying about identity (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 99%, past 85% since May 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"m-arenahard","name":"m-ArenaHard","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Aya Vision 32B. That’s expected for a test replaced by m-ArenaHard v2.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 78.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"m-arenahard-v2","name":"m-ArenaHard v2.0","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"m-arenahard-v2-1","name":"m-ArenaHard v2.1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"m-ifeval","name":"M-IFEval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","15 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in September 2024, for Qwen2.5-0.5B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Saturated. The best is 86.6%, past 85% since September 2024."],["import.importer","pass","We read the board on github.com: 15 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"m-rewardbench","name":"M-RewardBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","9 models we track, plus 14 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in October 2024, for Aya Expanse 32B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 83.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 9 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"m-tau-bench-cohere","name":"m-tau-bench (Cohere)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for Command A Reasoning. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 46.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"m-wildvision","name":"m-WildVision","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Aya Vision 32B. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 72%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"m3-bench","name":"M3-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"m3exam","name":"M3Exam","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in May 2024, for GPT-4o. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"m3gia","name":"M3GIA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (23 entries) and system cards (1 score)."],["scores.models","pass","18 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for STEP3-VL-10B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 78.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 17 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"m4singer","name":"M4Singer","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"m5hisdoc","name":"M5HisDoc","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (34 entries)."],["scores.models","pass","11 models we track, plus 23 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 64.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on TongGuOCR: 11 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ma-bench","name":"MA-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 57.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 20 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"macbench","name":"MaCBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 72.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ukgovernmentbeis.github.io: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"machiavelli","name":"MACHIAVELLI","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aypan17.github.io: 3 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"macrolens","name":"MacroLens","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"macroscopebench","name":"MacroscopeBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 83.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on macroscope.com: 57 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mad-bench","name":"MAD-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mad-eval","name":"MAD-Eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"madasr-2","name":"MADASR 2.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on sites.google.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"madasr","name":"MADASR 2023","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"made-1-0","name":"MADE 1.0","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"madinahqa","name":"MadinahQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (151 entries) and system cards (1 score)."],["scores.models","pass","40 models we track, plus 112 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Falcon-Arabic-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 73.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 52 rows for models we track and 112 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"madqa","name":"MADQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries) and launch posts (1 score)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","info","Saturated. The best is 91.8%, past 85% since November 2025."],["import.importer","pass","We read the board on huggingface.co: 27 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"maeb","name":"MAEB","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries) and system cards (1 score)."],["scores.models","pass","1 model we track, plus 23 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mteb-leaderboard-backend.hf.space: 1 row for models we track and 23 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"maestro-piano-transcription","name":"MAESTRO Piano Transcription","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on OpenCodePapers: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mafand-mt","name":"MAFAND-MT","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mafiabench","name":"MafiaBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on mafiabench.org: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mage-bench","name":"mage-bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Mini (#74)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mage-bench.com: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"magentic-marketplace","name":"Magentic Marketplace","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"magnatagatune","name":"MagnaTagATune","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"human-side-by-side-evaluation-microsoft","name":"MAI human side-by-side evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for MAI-Thinking-1."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"jailbreak-evaluation-suite-microsoft","name":"MAI internal jailbreak evaluation suite","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from June 2026, for MAI-Thinking-1."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"safety-and-over-refusal-evaluation-microsoft","name":"MAI internal safety and over-refusal evaluation","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from June 2026, for MAI-Thinking-1."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mai-thinking-1-blind-human-preference-microsoft","name":"MAI Thinking 1 blind human preference evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"make-data-count-finding-data-references","name":"Make Data Count: Finding Data References","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"makemepay-openai","name":"MakeMePay","summary":{"pass":5,"warn":1,"fail":0,"info":4,"text":"5 of 6 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for GPT-4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 57%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"makemesay-openai","name":"MakeMeSay","summary":{"pass":5,"warn":1,"fail":0,"info":4,"text":"5 of 6 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (10 scores)."],["scores.models","pass","10 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Grok 4.20."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4.20 (#61)."],["scores.headroom","pass","The best is 72%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"malaysian-sign-language-medical","name":"Malaysian Sign Language Medical Dataset","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"malicious-agentic-influence-campaigns-anthropic","name":"Malicious agentic influence campaigns (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"malicious-agentic-influence-campaigns-34-point-anthropic","name":"Malicious agentic influence campaigns, 34-point rubric (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"malicious-computer-use-anthropic","name":"Malicious computer use (Anthropic)","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (14 scores)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 99.4%, past 85% since November 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"malicious-use-of-claude-code-anthropic","name":"Malicious use of Claude Code (Anthropic)","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (15 scores)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 96.7%, past 85% since April 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"malt-freelancer-project-matching","name":"Malt Freelancer Project Matching","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Qwen3-Embedding-0.6B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 50.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"maltese-news-categories","name":"Maltese News Categories","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"malware-refusals-golden-set-openai","name":"Malware Refusals (Golden Set)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for GPT-5.1-Codex-Max. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 100%, past 85% since May 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"malware-refusals-synthetic-openai","name":"Malware Refusals (Synthetic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for codex-1. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 97%, past 85% since May 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mammalnet","name":"MammalNet","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on mammal-net.github.io: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mamubench","name":"MAMUBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mana-rp-bench","name":"Mana RP Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (38 entries)."],["scores.models","pass","32 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","4 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on mana.land: 32 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"management-consulting-tasks-openai","name":"Management Consulting Tasks (OpenAI)","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (7 entries) and launch posts (3 scores)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 43.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openai.com: 7 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"manager-coercion-benchmark","name":"Manager Coercion Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mandarin-minority-mt-tencent","name":"Mandarin-Minority translation (Tencent)","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Hunyuan-MT-7B. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 60.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mandos-run-003","name":"MANDOS Run 003","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 20 months ago, in March 2025, for GPT-4o. That’s expected for a test replaced by MANDOS Session 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 100%, past 85% since March 2025."],["import.importer","pass","We read the board on relayforge.tools: 1 row for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mandos-session-2","name":"MANDOS Session 2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Kimi K2.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Sonnet 4.6 (#57)."],["scores.headroom","info","Saturated. The best is 96.9%, past 85% since January 2025."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"maniladder","name":"ManiLadder","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on maniladder-benchmark.github.io that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"maniparena","name":"ManipArena","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"maniskill3","name":"ManiSkill3","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"manta-animal-welfare","name":"MANTA (Animal Welfare)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Grok 4.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mantic-vs-polymarket-backtest","name":"Mantic vs Polymarket backtest","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on blog.mantic.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mantis-eval","name":"Mantis-Eval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (21 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","11 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for LongCat-Flash-Omni."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 84.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"malt-metr","name":"Manually-Reviewed Agentic Labeled Transcripts","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"many-labs-2-text-replications","name":"Many Labs 2 text-based replications","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"many-shot-jailbreaking-anthropic","name":"Many-shot jailbreaking evaluation (Anthropic)","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Claude Opus 4. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4 (#109)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"manyih-bench","name":"ManyIH-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 60.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mapa-eur-lex","name":"MAPA EUR-Lex","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mapanything-dense-multi-view-reconstruction","name":"MapAnything dense multi-view reconstruction","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mapanything-single-view-calibration","name":"MapAnything single-view calibration","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mapp-salty-duck","name":"MAPP (Multi-Agent Path Planning)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for GPT-4.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4.1 (#138)."],["scores.headroom","pass","The best is 70.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on app.primeintellect.ai: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"maps-piano-transcription","name":"MAPS Piano Transcription","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"marathondocbench","name":"MarathonDocBench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries) and launch posts (1 score)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from October 2026, for Unlimited OCR 2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","pass","The best is 77.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"marineeval","name":"MarineEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","8 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for InternVL3-38B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 50.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on marineeval.hkustvgd.com: 8 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"marineevt","name":"MarineEVT","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","7 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 43.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on marineevt.hkustvgd.com: 10 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"markov-blanket-discovery","name":"Markov Blanket Discovery (shinpaku)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mars-bench","name":"MARS-Bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","17 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Seed1.8. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o4-mini (#95)."],["scores.headroom","info","Saturated. The best is 85.6%, past 85% since February 2026."],["import.importer","pass","We read the board on syuchin.github.io: 17 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"marsretrieval","name":"MarsRetrieval","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","3 models we track, plus 16 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"masakhaner","name":"MasakhaNER","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"masakhaner-2","name":"MasakhaNER 2.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"masakhanews","name":"MasakhaNEWS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"masakhapos","name":"MasakhaPOS","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#167)."],["scores.headroom","pass","The best is 62.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on AfroBench: 8 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mask","name":"MASK","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (32 entries), launch posts (2 scores) and system cards (18 scores)."],["scores.models","pass","33 models we track, plus 18 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 98.1%, past 85% since April 2026."],["import.importer","pass","We read the board on arxiv.org: 14 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"massive","name":"MASSIVE","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"massive-visualized-ir-benchmark","name":"Massive Visualized IR Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 11 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"massively-multilingual-speech-labeled-corpus","name":"Massively Multilingual Speech labeled corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mastermind-prime","name":"Mastermind (Prime Intellect)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for GLM-4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on app.primeintellect.ai: 22 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mastermind-information-gain","name":"Mastermind Information Gain","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mastermindeval","name":"MastermindEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for DeepSeek-R1-Distill-Qwen-1.5B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Phi-4 (#160)."],["scores.headroom","pass","The best is 53.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"matbench","name":"Matbench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Intern-S1-Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on matbench.materialsproject.org that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"matbert-aunp-ner","name":"MatBERT gold nanoparticle entity recognition","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"matcha","name":"MatCha","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","9 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for InternVL3-38B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 4 Maverick (#153)."],["scores.headroom","pass","The best is 68.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 54 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"materials-science-procedural-text-corpus","name":"Materials Science Procedural Text Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"math","name":"MATH","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries), launch posts (62 scores) and system cards (71 scores)."],["scores.models","pass","100 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Saturated. The best is 98.2%, past 85% since December 2024."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kangaroo-2025-matharena","name":"Math Kangaroo 2025 (MathArena)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","23 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.6 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 94.9%, past 85% since December 2025."],["import.importer","pass","We read the board on matharena.ai: 22 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"math-level-5","name":"MATH Level 5","summary":{"pass":5,"warn":1,"fail":0,"info":4,"text":"5 of 6 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from Epoch AI’s runs (94 models), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","96 models we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for LFM2-1.2B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 98.1%, past 85% since September 2024."],["import.importer","warn","The record links a board on epoch.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"math-side-tasking-monitor-evasion-openai","name":"Math Side-tasking Monitor Evasion","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"math-500","name":"MATH-500","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (54 entries), launch posts (48 scores) and system cards (46 scores)."],["scores.models","pass","104 models we track, plus 14 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Saturated. The best is 99.2%, past 85% since September 2024."],["import.importer","pass","We read the board on Vals AI: 46 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 6 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"math-vision-mathvision","name":"MATH-Vision (MathVision)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (153 entries), launch posts (48 scores) and system cards (32 scores)."],["scores.models","pass","133 models we track, plus 48 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Saturated. The best is 97.7%, past 85% since November 2025."],["import.importer","pass","We read the board on mathllm.github.io: 111 rows for models we track and 48 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 25 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"math-vr","name":"Math-VR","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries) and system cards (1 score)."],["scores.models","pass","26 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Qwen3-VL-235B-A22B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 66.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on math-vr.github.io: 28 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"math23k","name":"Math23K","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 2 years ago, in June 2024, for GLM-4. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen-14B (#214)."],["scores.headroom","info","Saturated. The best is 88.5%, past 85% since June 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"math24o","name":"Math24o","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 20 months ago, in March 2025, for QwQ-32B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1 (#118)."],["scores.headroom","info","Saturated. The best is 85.7%, past 85% since January 2025."],["import.importer","pass","We read the board on github.com: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"math401","name":"Math401","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for Qwen-14B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen-14B (#214)."],["scores.headroom","pass","The best is 70.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"matharena-apex-imo-usamo","name":"MathArena (Apex, IMO, USAMO)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (47 entries), launch posts (12 scores) and system cards (6 scores)."],["scores.models","pass","57 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Pro (#19)."],["scores.headroom","pass","The best is 81.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on matharena.ai: 48 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 3 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"matharena-apex-shortlist","name":"MathArena Apex Shortlist","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (38 entries), launch posts (4 scores) and system cards (2 scores)."],["scores.models","pass","40 models we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 98.4%, past 85% since February 2026."],["import.importer","pass","We read the board on matharena.ai: 38 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mathbench-opencompass","name":"MathBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","21 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for Qwen2-72B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 70.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on open-compass.github.io: 21 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mathcanvas-bench","name":"MathCanvas-Bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","17 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Seed2.0 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 61.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mathcanvas.github.io: 16 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mathclue","name":"MathCLUE","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","20 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-235B-A22B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 20 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alphaevolve-mathematical-problems-67","name":"Mathematical Problem Repository for AlphaEvolve","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mathif","name":"MathIF","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","12 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-0.6B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 50.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 12 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"matholympiadbench","name":"MathOlympiadBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mathqa","name":"MathQA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry) and system cards (1 score)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Apertus 70B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 33.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on math-qa.github.io: 1 entry at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mathtutorbench","name":"MathTutorBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","12 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Apertus 1.5 8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eth-lre.github.io: 12 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mathverse","name":"MathVerse","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries), launch posts (7 scores) and system cards (11 scores)."],["scores.models","pass","22 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 89.7%, past 85% since September 2025."],["import.importer","pass","We read the board on mathverse-cuhk.github.io: 9 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mathvision-wildphoto-qwen","name":"MathVision-Wildphoto","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Qwen3-VL-235B-A22B. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 63.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mathvista","name":"MathVista","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (78 entries), launch posts (63 scores) and system cards (42 scores)."],["scores.models","pass","100 models we track, plus 35 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K2.5 (#76)."],["scores.headroom","info","Saturated. The best is 90.7%, past 85% since April 2025."],["import.importer","pass","We read the board on mathvista.github.io: 45 rows for models we track and 33 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 14 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"matsci-nlp","name":"MatSci-NLP","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"matscibench","name":"MatSciBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 64%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"matter-of-fact","name":"Matter-of-Fact","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for o4-mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","pass","The best is 72%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mattoolbench","name":"MatToolBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 51.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mattoolbench.github.io: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mattools","name":"MatTools","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","4 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Gemini 2.0 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","pass","The best is 81.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"maud","name":"MAUD","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"maverix","name":"MAVERIX","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","15 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Nova 2 Omni Preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4 (#86)."],["scores.headroom","pass","The best is 66.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 48 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mawarith","name":"MAWARITH","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on sites.google.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mawps","name":"MAWPS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"maxife","name":"MAXIFE","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (11 scores)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.7-Max (#49)."],["scores.headroom","info","Saturated. The best is 89.2%, past 85% since February 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"maxm","name":"MaXM","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Aya Vision 32B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"maxxbench","name":"MaxxBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 76.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on maxxbench.unlap.ai: 17 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"maydan-hf-rl-sampler","name":"Maydan HF RL Sampler","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"maze-bench","name":"Maze Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (58 entries)."],["scores.models","pass","58 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 24%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mazebench.com: 192 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mbabench","name":"MBABench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","7 models we track, plus 5 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Qwen 3.6 Plus. That’s expected for a test replaced by MBABench 2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mbabench.org: 7 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mbabench-2","name":"MBABench 2.0","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","6 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 15.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snorkel.ai: 15 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multistate-bar-examination","name":"MBE (Multistate Bar Examination)","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 2 years ago, in March 2024, for Claude 3 Haiku. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Saturated. The best is 85%, past 85% since March 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mbench-motion","name":"MBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mbpp","name":"MBPP","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (89 entries), launch posts (38 scores) and system cards (55 scores)."],["scores.models","pass","93 models we track, plus 54 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Saturated. The best is 95.5%, past 85% since November 2023."],["import.importer","pass","We read the board on EvalPlus: 35 rows for models we track and 54 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 11 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mbpp-plus","name":"MBPP+","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (89 entries), launch posts (12 scores) and system cards (10 scores)."],["scores.models","pass","50 models we track, plus 54 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Granite 4.1 30B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Saturated. The best is 86.2%, past 85% since April 2025."],["import.importer","pass","We read the board on evalplus.github.io: 35 rows for models we track and 54 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mbti-post-classification-kunumi","name":"MBTI Post Classification (Kunumi)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mbxp","name":"MBXP","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for Seed Diffusion Preview. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 78.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mc-bench","name":"MC-Bench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mc-taco","name":"MC-TACO","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mceval","name":"McEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries) and system cards (1 score)."],["scores.models","pass","7 models we track, plus 16 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Hunyuan-A13B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 65.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mceval.github.io: 6 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mcp-atlas","name":"MCP Atlas","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries), launch posts (44 scores) and system cards (22 scores)."],["scores.models","pass","64 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 90.7%, past 85% since July 2026."],["import.importer","pass","We read the board on labs.scale.com: 37 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 20 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mcp-prompt-injection-anthropic","name":"MCP prompt injection evaluation (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 13 months ago, in October 2025, for Claude Haiku 4.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Sonnet 4.5 (#81)."],["scores.headroom","info","Saturated. The best is 92.5%, past 85% since September 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mcp-bench","name":"MCP-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","16 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 16 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mcp-safetybench","name":"MCP-SafetyBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","10 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 21.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mcp-sim-physics-simulation","name":"MCP-SIM","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mcp-universe","name":"MCP-Universe","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","32 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 45.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mcp-universe.github.io: 34 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mcpmark","name":"MCPMark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries), launch posts (10 scores) and system cards (3 scores)."],["scores.models","pass","37 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Saturated. The best is 94.5%, past 85% since July 2026."],["import.importer","pass","We read the board on mcpmark.ai: 39 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mcpmark-verified","name":"MCPMark Verified","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 96.1%, past 85% since April 2026."],["import.importer","pass","We read the board on mcpmark.ai: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mcptox","name":"MCPTox","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","9 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-14B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mcs-bench","name":"MCS-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (37 entries)."],["scores.models","pass","36 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 36 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mcstasbench","name":"McStasBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.5 Flash-Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 5 (#35)."],["scores.headroom","pass","The best is 43.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"md-coedit-bench-tencent","name":"MdCoEditBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mdmr-bench","name":"MDMR-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","7 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Flash Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 81.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mdolly","name":"mDolly","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mdpbench","name":"MDPBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","13 models we track, plus 29 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen3.8-Flash-Next."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 13 rows for models we track and 29 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"measeval","name":"MeasEval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries)."],["scores.models","pass","20 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on competitions.codalab.org: 20 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"measurebench","name":"MeasureBench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"measurebench-real-world","name":"MeasureBench Real-world","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","19 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 41.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on flageval-baai.github.io: 19 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"measurebench-synthetic","name":"MeasureBench Synthetic","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","19 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 39.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on flageval-baai.github.io: 19 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meccano-action-recognition","name":"MECCANO Action Recognition","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meceng","name":"MecEng","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","20 models we track, plus 14 other entries on its leaderboard."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Saturated. The best is 91.4%, past 85% since December 2025."],["import.importer","pass","We read the board on github.com: 20 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"med-easi","name":"Med-EASi","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"med-halt","name":"Med-HALT","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","6 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Llama 2-13B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-70B (#210)."],["scores.headroom","pass","The best is 42.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 6 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medagentbench","name":"MedAgentBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Gemini 2.0 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3-mini (#124)."],["scores.headroom","pass","The best is 69.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medalign","name":"MedAlign","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medbench-russian-closed-test","name":"MedBench (Russian closed-test platform)","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on medbench.ru that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medbench-opencompass","name":"MedBench 1.0 (OpenCompass)","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on shlab.org.cn: 10 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medbench-opencompass-2","name":"MedBench 2.0 (OpenCompass)","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medbench-opencompass-3","name":"MedBench 3.0 (OpenCompass)","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medbench-opencompass-4","name":"MedBench 4.0 (OpenCompass)","summary":{"pass":1,"warn":3,"fail":0,"info":6,"text":"1 of 4 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medbench-opencompass-5","name":"MedBench 5.0 (OpenCompass)","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Qwen3.7-Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on medbench.opencompass.org.cn: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medbullets","name":"Medbullets","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Llama 3-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-70B (#182)."],["scores.headroom","pass","The best is 68.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 16 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medcalc-bench-original","name":"MedCalc-Bench (original)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in April 2024, for Llama 3-70B. That’s expected for a test replaced by MedCalc-Bench Verified."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3-70B (#182)."],["scores.headroom","pass","The best is 39.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medcalc-bench-verified","name":"MedCalc-Bench Verified","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries) and system cards (1 score)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Saturated. The best is 86.2%, past 85% since February 2026."],["import.importer","pass","We read the board on Anthropic: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medchembench-openai","name":"MedChemBench (OpenAI)","summary":{"pass":11,"warn":1,"fail":0,"info":0,"text":"11 of 12 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries), launch posts (4 scores) and system cards (1 score)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-Rosalind-6."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 83.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cdn.openai.com: 61 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","3 of 4 pairs differ by more than 10 points, the largest for GPT-6 Astra (−35 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-medcode","name":"MedCode","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (97 entries)."],["scores.models","pass","97 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Small."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 63.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 108 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medconceptsqa","name":"MedConceptsQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","2 models we track, plus 17 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-3.5 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-3.5 Turbo (#200)."],["scores.headroom","pass","The best is 61.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meddialog","name":"MedDialog","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meddoprof","name":"MEDDOPROF","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medec","name":"MEDEC","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","3 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Gemini 2.0 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","pass","The best is 69.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 3 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medgemma-lab-report-conversion-google","name":"MedGemma lab report conversion","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medgenvidqa-2026-visual-answer-localization","name":"MedGenVidQA 2026 Visual Answer Localization","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on codabench.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medhallu","name":"MedHallu","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","1 model we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Gemma 2 2B IT."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on medhallu.github.io: 2 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medheb","name":"MedHEB","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","2 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Qwen3-VL-Embedding-2B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medhelm","name":"MedHELM","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on crfm.stanford.edu: 13 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"media-bias-identification-benchmark","name":"Media Bias Identification Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"media-frames-corpus","name":"Media Frames Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mediaperf","name":"MediaPerf","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","16 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Nemotron 3 Nano Omni 30B-A3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mediaperf.org: 16 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medical-bench-turing","name":"Medical Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 88%, past 85% since August 2026."],["import.importer","pass","We read the board on bench.turing.com: 9 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medical-chronology-llm-benchmark","name":"Medical Chronology LLM Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Pro (#19)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mksap","name":"Medical Knowledge Self-Assessment Program (MKSAP)","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 3 years ago, in March 2023, for GPT-4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","pass","The best is 75%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medical-paper-blog","name":"Medical Paper-Blog Dataset","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"lm-eval-med-prescriptions","name":"Medical Prescription Selection (lm-evaluation-harness)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omi-medical-speech-benchmark","name":"Medical Speech-to-Text Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","5 models we track, plus 24 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Muse Voice Transcribe 1.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on omi.health: 6 rows for models we track and 24 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medicat","name":"MedICaT","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medicationqa","name":"MedicationQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medicinal-chemistry-anthropic","name":"Medicinal Chemistry (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 65.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mediqa-2019-question-answering","name":"MEDIQA 2019 Question Answering","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mediqa-synur","name":"MEDIQA-SYNUR","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medlfqa","name":"MedLFQA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","3 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Claude 3 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Sonnet (#187)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medmcqa","name":"MedMCQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries) and system cards (1 score)."],["scores.models","pass","1 model we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 77%, under 85%, so it still separates models."],["import.importer","pass","We read the board on medmcqa.github.io: 7 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medmentions","name":"MedMentions","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medphysbench","name":"MedPhysBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since August 2025."],["import.importer","pass","We read the board on github.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medphysbench-kq","name":"MedPhysBench Knowledge Questions","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medprocner","name":"MedProcNER","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ceur-ws.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medpt","name":"MedPT","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medqa","name":"MedQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (91 entries), launch posts (1 score) and system cards (4 scores)."],["scores.models","pass","73 models we track, plus 19 other entries on its leaderboard."],["scores.recent","info","The newest came 20 months ago, in March 2025, for DeepSeek-V3. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V3 (#141)."],["scores.headroom","info","Saturated. The best is 89.4%, past 85% since May 2024."],["import.importer","pass","We read the board on Stanford HELM: 72 rows for models we track and 19 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medqa-bias-vals","name":"MedQA Bias Evaluation","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (84 entries)."],["scores.models","pass","83 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 96.5%, past 85% since July 2024."],["import.importer","pass","We read the board on Vals AI: 94 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medqa-mainland-china","name":"MedQA Mainland China","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 84%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medqa-taiwan","name":"MedQA Taiwan","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Saturated. The best is 91%, past 85% since August 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medqa-usmle-5-options","name":"MedQA USMLE (5 options)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Saturated. The best is 86%, past 85% since August 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medqa-followup","name":"MedQA-Followup","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","12 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Claude Opus 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 0%, under 85%, so it still separates models."],["import.importer","pass","We read the board on bmanczak.github.io: 14 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medquad","name":"MedQuAD","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on pmc.ncbi.nlm.nih.gov: 4 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medreadme","name":"MedReadMe","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medsafe-dx-0-3","name":"MedSafe-Dx v0.3","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","19 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 19 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medsafetybench","name":"MedSafetyBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (30 entries)."],["scores.models","pass","10 models we track, plus 20 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-70B (#177)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on proceedings.neurips.cc: 10 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-medscribe","name":"MedScribe","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (99 entries)."],["scores.models","pass","98 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Small."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 91.4%, past 85% since October 2025."],["import.importer","pass","We read the board on vals.ai: 109 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medsecid","name":"MedSecId","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medtext","name":"MedText","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medvidqa-miqg-2023","name":"MedVidQA 2023 Medical Instructional Question Generation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codalab.lisn.upsaclay.fr: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medvidqa-vcval-2023","name":"MedVidQA 2023 Video Corpus Visual Answer Localization","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medvidqa-qfisc-2024","name":"MedVidQA 2024 Query-Focused Instructional Step Captioning","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medvidqa-vcval-2024","name":"MedVidQA 2024 Video Corpus Visual Answer Localization","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medxpertqa","name":"MedXpertQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","6 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for MAI-Thinking-1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark (#54)."],["scores.headroom","pass","The best is 52.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medxpertqa-mm-extended-mercor","name":"MedXpertQA MM Extended","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","41 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 71.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mercor.com: 42 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"medxpertqa-mm","name":"MedXpertQA-MM","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries), launch posts (20 scores) and system cards (7 scores)."],["scores.models","pass","30 models we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 80.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on medxpertqa.github.io: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meen-361-conceptual-question-grading","name":"MEEN 361 Conceptual Question Grading","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meeseeks","name":"Meeseeks","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 13 months ago, in October 2025, for LongCat-Flash-Omni. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 43%, under 85%, so it still separates models."],["import.importer","warn","The record links a board on github.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meeseeks-utility-rate","name":"Meeseeks Utility Rate","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for DeepSeek-V3.1. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4 (#109)."],["scores.headroom","info","Saturated. The best is 90.5%, past 85% since April 2025."],["import.importer","pass","We read the board on arxiv.org: 16 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mega-bench","name":"MEGA-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries) and system cards (1 score)."],["scores.models","pass","23 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Qwen2.5-VL-72B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (October 2024) (#150)."],["scores.headroom","pass","The best is 54.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tiger-ai-lab.github.io: 22 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"megagem","name":"MegaGem","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"megaminx-solver","name":"Megaminx solver","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"megaoath-forecasting-survey","name":"MegaOath Forecasting Survey","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mela","name":"MELA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meld","name":"MELD","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mem-gallery","name":"Mem-Gallery","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (52 entries)."],["scores.models","pass","52 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 91 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mementos","name":"Mementos","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","1 model we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for GPT-4V."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 40.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"memeye","name":"MemEye","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Mini (#74)."],["scores.headroom","pass","The best is 49.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on minghokwok.github.io: 50 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"memgui-bench","name":"MemGUI-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","15 models we track, plus 13 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 73.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on memgui-bench.github.io: 21 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"memlens","name":"MemLens","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (25 entries)."],["scores.models","pass","24 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 53.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 24 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"memory-discipline","name":"Memory Discipline","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries)."],["scores.models","pass","35 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 99.7%, past 85% since May 2025."],["import.importer","pass","We read the board on kaggle.com: 36 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"memoryagentbench","name":"MemoryAgentBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5 mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 mini (#92)."],["scores.headroom","pass","The best is 60.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"memorybench-meta","name":"MemoryBench (Meta)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder Full."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"memorytrap-bench-php-arrays-go-maps","name":"MemoryTrap Bench: PHP arrays, Go maps","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"memtrack","name":"MEMTRACK","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 61%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"memtrapbench","name":"MemTrapBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 11 months ago, in December 2025, for Gemini 3 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","info","Saturated. The best is 85.2%, past 85% since December 2025."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"menagerie-program-grading","name":"Menagerie: A Dataset of Graded Programming Assignments","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ment-sns","name":"MENT-SNS","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for GLM-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GLM-5 (#89)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mentalbench-dsm","name":"MentalBench (Psychiatric Diagnosis)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","19 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for GPT-5.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.1 (#66)."],["scores.headroom","pass","The best is 62.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaist-mlplab.github.io: 19 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mentalbench-100k","name":"MentalBench-100k","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mentalchat16k-counseling","name":"MentalChat16K Counseling Evaluation","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","5 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in December 2023, for Mistral-7B-Instruct-v0.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Mixtral 8x7B (#199)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 20 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mentalhealthbench","name":"MentalHealthBench","summary":{"pass":11,"warn":1,"fail":0,"info":0,"text":"11 of 12 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries) and system cards (7 scores)."],["scores.models","pass","16 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 57.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cdn.openai.com: 12 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meqsum","name":"MeQSum","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mer-unibench","name":"MER-UniBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","15 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 15 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mera","name":"MERA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (209 entries)."],["scores.models","pass","102 models we track, plus 107 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mera.a-ai.ru: 277 rows for models we track and 107 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"merchantbench","name":"MerchantBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on air.1688.com: 16 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mercury-code-efficiency","name":"Mercury","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","4 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for CodeQwen1.5-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is StarCoder 2 15B (#224)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mermaidseqbench","name":"MermaidSeqBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mesa","name":"MESA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GR00T N1.6."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 50.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on pairlab.github.io: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"met-meme","name":"MET-Meme","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"action-belief-coherence-meta","name":"Meta action-belief coherence probe","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Saturated. The best is 89.1%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-cyber-misuse-refusal-meta","name":"Meta agentic cyber misuse refusal","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"user-traffic-replay-evaluation-meta","name":"Meta agentic evaluations grounded in user traffic","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meta-agents-research-environments-are-gaia2","name":"Meta Agents Research Environments (ARE) / Gaia2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Glimmer."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 77.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 16 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meta-biodesign-proxy","name":"Meta BioDesign Proxy","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meta-bioknowledge-proxy","name":"Meta BioKnowledge Proxy","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 13 months ago, in October 2025, for Code World Model (CWM). Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meta-bioprotocol-proxy","name":"Meta BioProtocol Proxy","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 13 months ago, in October 2025, for Code World Model (CWM). Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 43.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"catastrophic-cyber-misuse-refusal-meta","name":"Meta catastrophic cyber misuse refusal","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cb-high-risk-bottlenecks-meta","name":"Meta chemical and biological high-risk bottleneck evaluations","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from June 2026, for Muse Spark."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chemical-agents-refusal-meta","name":"Meta chemical-agent refusal evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since June 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"code-fetch-under-user-requests-meta","name":"Meta Code-Fetch Under User Requests","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"curated-ctf-set-meta","name":"Meta curated CTF set","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Muse Spark 1.1."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.1 (#48)."],["scores.headroom","info","Saturated. The best is 89.9%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyber-false-refusal-meta","name":"Meta cyber false-refusal benchmark","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyber-misuse-chat-meta","name":"Meta Cyber Misuse Chat","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Muse Spark 1.1."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.1 (#48)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cyber-misuse-direct-requests-meta","name":"Meta cyber misuse request set","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for Muse Spark."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark (#54)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"petri-bloom-holistic-audit-meta","name":"Meta holistic adversarial audit with Petri Bloom","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"instructed-sandbagging-eval-meta","name":"Meta instructed selective performance evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"internal-agent-hallucination-eval-meta","name":"Meta internal agent hallucination evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sycophancy-eval-meta","name":"Meta internal sycophancy evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompted-evaluation-awareness-meta","name":"Meta prompted evaluation awareness","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for Muse Spark."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark (#54)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"targeted-agentic-evaluations-meta","name":"Meta targeted agentic evaluations","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"training-stage-provenance-meta","name":"Meta training-stage provenance probe","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"underspecified-intent-eval-meta","name":"Meta Underspecified Intent evaluation","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for Muse Spark."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark (#54)."],["scores.headroom","pass","The best is 71.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meta-album","name":"Meta-Album","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meta-dataset-google","name":"Meta-Dataset","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (25 entries)."],["scores.models","pass","25 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 25 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meta-gui","name":"META-GUI","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on x-lance.github.io: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"meta-world","name":"Meta-World","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"metabench","name":"metabench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (464 entries)."],["scores.models","pass","5 models we track, plus 459 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for Yi-1.5-9B-Chat."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Mixtral 8x22B (#189)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 459 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"metagenomicsbench","name":"MetagenomicsBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 62.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 43 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"metal-arm-harness","name":"metal-arm-harness","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"metr-capability-index","name":"METR Capability Index","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries)."],["scores.models","pass","28 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on metr.org: 28 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"metr-data-deduplication","name":"METR data deduplication","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (4 entries) and system cards (3 scores)."],["scores.models","pass","3 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Claude Opus 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4 (#109)."],["scores.headroom","pass","The best is 32.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on metr.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cot-faithfulness-monitorability-metr","name":"METR Faithfulness and Monitorability Experiments","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"metr-general-autonomy-suite","name":"METR general autonomy task suite","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for o1."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"metr-pre-deployment-ai-r-and-d-tasks","name":"METR pre-deployment AI R&D tasks","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"side-task-monitorability-metr","name":"METR side task monitorability evaluation","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"metr-time-horizon","name":"METR Time Horizon","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries), system cards (5 scores) and claims Epoch AI collected (26 models)."],["scores.models","pass","39 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as minutes, with no fixed ceiling."],["import.importer","pass","We read the board on metr.org: 31 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"metr-time-horizon-1-1","name":"METR Time Horizon 1.1","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Mythos Preview. That’s expected for a test replaced by METR Time Horizon 2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as minutes, with no fixed ceiling."],["import.importer","pass","We read the board on metr.org: 26 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"metr-time-horizon-2","name":"METR Time Horizon 2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as minutes, with no fixed ceiling."],["import.importer","pass","We read the board on metr.org: 11 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"metr-time-horizon-suite","name":"METR Time Horizon suite","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","25 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Claude Opus 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","info","Scored as minutes, with no fixed ceiling."],["import.importer","pass","We read the board on metr.org: 25 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"metrollm-bench","name":"MetroLLM-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","19 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on continker.ai: 21 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mevis","name":"MeViS","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on raw.githubusercontent.com: 1 entry at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mewc","name":"MEWC","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for MiniMax-M2.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 74.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mewc-v2-minimax","name":"MEWC v2","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for MiniMax-M2.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 63.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mfollowir","name":"mFollowIR","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries)."],["scores.models","pass","1 model we track, plus 17 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for Mistral-7B-Instruct-v0.1."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mgsm","name":"MGSM","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (64 entries), launch posts (33 scores) and system cards (41 scores)."],["scores.models","pass","113 models we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Saturated. The best is 95.2%, past 85% since May 2023."],["import.importer","pass","We read the board on Vals AI: 75 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 8 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mgsm-rev2","name":"MGSM-Rev2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","8 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 13 months ago, in October 2025, for Claude Haiku 4.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 99.3%, past 85% since March 2025."],["import.importer","pass","We read the board on aclanthology.org: 8 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mhpp","name":"MHPP","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","24 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Seed-Coder-8B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","pass","The best is 68.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sparksofagi.github.io: 23 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mia-bench","name":"MIA-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (34 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","27 models we track, plus 9 other entries on its leaderboard."],["scores.recent","info","The newest came 10 months ago, in January 2026, for STEP3-VL-10B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Saturated. The best is 92.7%, past 85% since November 2023."],["import.importer","pass","We read the board on arxiv.org: 45 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"micro1-medical-index","name":"micro1 Medical Index","summary":{"pass":9,"warn":2,"fail":0,"info":1,"text":"9 of 11 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","6 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 71%, under 85%, so it still separates models."],["import.importer","pass","We read the board on micro1.ai: 10 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multimodal-rai-internal-microsoft","name":"Microsoft internal multimodal RAI benchmark","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 20 months ago, in March 2025, for Phi-4-multimodal. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rai-grounding-microsoft","name":"Microsoft RAI benchmark: Grounding","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for Phi-4. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Phi-4 (#160)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rai-harmful-content-continuation-microsoft","name":"Microsoft RAI benchmark: Harmful Content Continuation (DR-3)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for Phi-4. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Phi-4 (#160)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rai-harmful-content-summarization-microsoft","name":"Microsoft RAI benchmark: Harmful Content Summarization (DR-3)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for Phi-4. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Phi-4 (#160)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rai-jailbreak-microsoft","name":"Microsoft RAI benchmark: Jailbreak (DR-1)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for Phi-4. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Phi-4 (#160)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rai-third-party-harm-microsoft","name":"Microsoft RAI benchmark: Third Party Harm (DR-1)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for Phi-4. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Phi-4 (#160)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rai-ungroundedness-microsoft","name":"Microsoft RAI benchmark: Ungroundedness","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 2 years ago, in August 2024, for Phi-3.5-MoE. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is phi-3-small 7.4B (#193)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"microvqa","name":"MicroVQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries), launch posts (3 scores) and system cards (2 scores)."],["scores.models","pass","22 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Intern-S2-397B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 69.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on jmhb0.github.io: 19 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mieb","name":"MIEB","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries)."],["scores.models","pass","41 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mteb-leaderboard-backend.hf.space: 41 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mieb-lite","name":"MIEB Lite","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (51 entries) and system cards (1 score)."],["scores.models","pass","1 model we track, plus 50 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mteb-leaderboard-backend.hf.space: 1 row for models we track and 50 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mijabench","name":"MiJaBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","11 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 58.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 11 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"miklos-schweitzer-2025","name":"Miklós Schweitzer 2025","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 13 months ago, in October 2025, for GPT-5 Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 Pro (#64)."],["scores.headroom","info","Saturated. The best is 90%, past 85% since October 2025."],["import.importer","pass","We read the board on matharena.ai: 1 row for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"milu","name":"MILU","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (27 entries), launch posts (1 score) and system cards (10 scores)."],["scores.models","pass","19 models we track, plus 19 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 94.1%, past 85% since February 2026."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 19 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mimic-cxr-classification","name":"MIMIC-CXR (classification)","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on CodeSOTA: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mimic-cxr-report-generation","name":"MIMIC-CXR report generation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on SOTA2 Research: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mimic-iii-report-summarization","name":"MIMIC-III Report Summarization","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mimic-iv-bhc","name":"MIMIC-IV-BHC","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mimo-code-bench","name":"MiMo Code Bench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (8 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from September 2026, for MiMo-V2.6 Pro."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 68.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mimo.xiaomi.com: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mimo-coding-bench","name":"MiMo Coding Bench","summary":{"pass":9,"warn":2,"fail":0,"info":1,"text":"9 of 11 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and launch posts (2 scores)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for MiMo-V2.5-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 77.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mimo.xiaomi.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mimo-cyber-bench","name":"MiMo Cyber Bench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for MiMo-V2.6 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 81.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mimo.xiaomi.com: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mimo-visual-coding","name":"MiMo Visual Coding","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (9 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for MiMo-V2.6 Pro."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 82.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mimo.xiaomi.com: 9 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mind-the-gap","name":"Mind the Gap","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","6 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.5 Flash (#44)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mind2web","name":"Mind2Web","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Qwen3-Coder-480B-A35B. That’s expected for a test replaced by Mind2Web 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 55.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mind2web-2","name":"Mind2Web 2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","3 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for o3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 28%, under 85%, so it still separates models."],["import.importer","pass","We read the board on osu-nlp-group.github.io: 3 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mind2web-sc","name":"Mind2Web-SC","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 23 months ago, in December 2024, for Llama 3.3 70B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.3 70B (#171)."],["scores.headroom","info","Saturated. The best is 93%, past 85% since December 2024."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mindcube","name":"MindCube","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries), launch posts (1 score) and system cards (3 scores)."],["scores.models","pass","13 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Sonnet 4 (#113)."],["scores.headroom","info","Saturated. The best is 89.9%, past 85% since January 2026."],["import.importer","pass","We read the board on mind-cube.github.io: 9 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mindeval","name":"MindEval","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mheval.slingshotai.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mindeval-2","name":"MindEval2","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mheval.slingshotai.com: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mindgames","name":"MindGames","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"minds-14","name":"MInDS-14","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mindworldbench","name":"MindWorldBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mineamongus","name":"MineAmongUs","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen3.6 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 81.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 48 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"minebench","name":"MineBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (83 entries)."],["scores.models","pass","81 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on minebench.ai: 81 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"minecraft-bed-wars-yc","name":"Minecraft Bed Wars (YC Labs)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"minecraft-builder-action-prediction","name":"Minecraft Builder Action Prediction","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"minedojo","name":"MineDojo","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Seed2.0 Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 49%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"minerva-video","name":"MINERVA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries), launch posts (3 scores) and system cards (5 scores)."],["scores.models","pass","13 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 70.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"minerva-cultural","name":"MINERVA-Cultural","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 42.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"minerva-ego","name":"Minerva-Ego","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 37.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ming-freeform-audio-edit","name":"Ming-Freeform-Audio-Edit","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mini-amusement-parks","name":"Mini Amusement Parks","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on maps.skyfall.ai: 46 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"minictx-v2","name":"miniCTX-v2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries) and launch posts (1 score)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","pass","The best is 81.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cmu-l3.github.io: 4 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"minif2f","name":"miniF2F","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (5 scores) and system cards (4 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for Leanstral 1.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 100%, past 85% since April 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"minimal-linuxbench-anthropic","name":"Minimal-LinuxBench (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"minimax-tts-multilingual-test-set","name":"MiniMax TTS Multilingual Test Set","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for Ming-3.0-tts-0.5B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"minists","name":"MiniStS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"miniwob-plus-plus","name":"MiniWoB++","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on BrowserGym: 10 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"minteval","name":"MINTEval","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen 3.6 35B-A3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.6 35B-A3B (#102)."],["scores.headroom","pass","The best is 21%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mir-1k","name":"MIR-1K","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mira-bench-moonshot","name":"MIRA Bench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 64.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mirage-mmmt","name":"MIRAGE-MMMT","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Llama 4 Maverick."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 65.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mirage-benchmark.github.io: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mirage-mmst","name":"MIRAGE-MMST","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","22 models we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for GPT-4.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mirage-benchmark.github.io: 22 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"miragebench","name":"MirageBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mirb","name":"MIRB","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries) and system cards (1 score)."],["scores.models","pass","8 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for InternVL3.5-241B-A28B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 73%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mirex-2026-cmi-rewardbench","name":"MIREX 2026 Music Evaluation via CMI-RewardBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on futuremirex.com: 14 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mirex-2026-music-performance-difficulty-prediction","name":"MIREX 2026 Music Performance Difficulty Prediction","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on futuremirex.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mirrorcode","name":"MirrorCode","summary":{"pass":7,"warn":1,"fail":0,"info":2,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from Epoch AI’s runs (41 models)."],["scores.models","pass","41 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","8 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 97.8%, past 85% since July 2026."],["import.importer","warn","The record links a board on epoch.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mirrordojo-reflection","name":"MirrorDojo","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Beam 1.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Beam 1.0 (#67)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mirrordyn-reflection","name":"MirrorDyn","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Beam 1.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Beam 1.0 (#67)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realistic-work-environments-misalignment-openai","name":"Misaligned Behavior in Realistic Work Environments","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"misbehavior-thinking-machines","name":"Misbehavior (Thinking Machines)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"misc-ctf-anthropic","name":"Misc CTF challenges (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since November 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"misconceptdialogue","name":"MisconceptDialogue","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Gemini 3.1 Flash-Lite. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.1 Flash-Lite (#96)."],["scores.headroom","info","Saturated. The best is 87.8%, past 85% since October 2025."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"misguided-attention","name":"Misguided Attention","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (50 entries)."],["scores.models","pass","45 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemma 3n E4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","pass","The best is 73.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cpldcpu.github.io: 53 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mishap-bench","name":"MISHAP-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","8 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.7 Flash (#26)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 32 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"missing-context-hallucinations-anthropic","name":"Missing-context hallucinations (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"missionreport","name":"MissionReport","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 64.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on micro1.ai: 11 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"unsafe-prompts-175-mistral","name":"Mistral 175 unsafe prompts","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 3 years ago, in October 2023, for Mistral 7B v0.1. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Mistral 7B v0.1 (#212)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since October 2023."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"function-calling-internal-mistral","name":"Mistral internal function calling benchmark","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Magistral Medium 1.0. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 87.4%, past 85% since June 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"self-reflection-moderation-mistral","name":"Mistral self-reflection moderation set","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 3 years ago, in October 2023, for Mistral 7B v0.1. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mit-adobe-fivek","name":"MIT-Adobe FiveK","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on people.csail.mit.edu: 5 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mitosis-bench","name":"Mitosis Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 32.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on dylantirandaz.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mivas-bench","name":"MIVAS Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","4 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Grok Voice 3.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 56.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on research.getbluejay.ai: 4 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mixer-6-speech","name":"Mixer 6 Speech","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mixeval","name":"MixEval","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (71 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","36 models we track, plus 37 other entries on its leaderboard."],["scores.recent","info","The newest came 16 months ago, in July 2025, for AFM-4.5B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","info","Saturated. The best is 89.9%, past 85% since February 2024."],["import.importer","pass","We read the board on mixeval.github.io: 34 rows for models we track and 37 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mizan-persian-llm-leaderboard","name":"MIZAN: A Persian LLM Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries)."],["scores.models","pass","35 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mcinext-mizan-llm-leaderboard.hf.space: 39 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mideind-icelandic-arc-challenge","name":"Miðeind Icelandic ARC-Challenge","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (90 entries)."],["scores.models","pass","78 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 95.2%, past 85% since February 2024."],["import.importer","pass","We read the board on mideind-icelandic-llm-leaderboard.static.hf.space: 80 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mideind-icelandic-inflection","name":"Miðeind Icelandic Case Declension","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (91 entries)."],["scores.models","pass","79 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 98%, past 85% since May 2024."],["import.importer","pass","We read the board on mideind-icelandic-llm-leaderboard.static.hf.space: 81 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mideind-icelandic-case-government","name":"Miðeind Icelandic Case Government","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (62 entries)."],["scores.models","pass","61 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since February 2026."],["import.importer","pass","We read the board on mideind-icelandic-llm-leaderboard.static.hf.space: 103 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mideind-icelandic-generative-quality","name":"Miðeind Icelandic Generative Quality","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (62 entries)."],["scores.models","pass","59 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 87.7%, past 85% since November 2026."],["import.importer","pass","We read the board on mideind-icelandic-llm-leaderboard.static.hf.space: 101 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mideind-icelandic-gec","name":"Miðeind Icelandic Grammar Correction","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (62 entries)."],["scores.models","pass","61 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 83.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mideind-icelandic-llm-leaderboard.static.hf.space: 103 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mideind-icelandic-ged","name":"Miðeind Icelandic Grammatical Error Detection","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (92 entries)."],["scores.models","pass","79 models we track, plus 13 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 91.5%, past 85% since June 2026."],["import.importer","pass","We read the board on mideind-icelandic-llm-leaderboard.static.hf.space: 81 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mideind-icelandic-idioms","name":"Miðeind Icelandic Idioms","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (62 entries)."],["scores.models","pass","61 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 85.7%, past 85% since November 2026."],["import.importer","pass","We read the board on mideind-icelandic-llm-leaderboard.static.hf.space: 103 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mideind-icelandic-popular-culture","name":"Miðeind Icelandic Popular Culture","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (62 entries)."],["scores.models","pass","61 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 39.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mideind-icelandic-llm-leaderboard.static.hf.space: 103 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mideind-icelandic-terminology","name":"Miðeind Icelandic Terminology","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (62 entries)."],["scores.models","pass","61 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 78%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mideind-icelandic-llm-leaderboard.static.hf.space: 103 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mj-bench","name":"MJ-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","8 models we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for InternVL-Chat-V1-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 44.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 8 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mk-bench-dynamic-scheduling","name":"MK-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","5 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3.2 (#91)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mka","name":"MKA","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 56.5%, under 85%, so it still separates models."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mkqa","name":"MKQA","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ml-dev-bench","name":"ML-Dev-Bench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mlagentbench","name":"MLAgentBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in December 2023, for Gemini 1.0 Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 18.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mlcommons-ai-safety-0-5","name":"MLCommons AI Safety Benchmark v0.5","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mlcommons-ailuminate","name":"MLCommons AILuminate","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","10 models we track, plus 17 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GLM-5.3-Flash (#53)."],["scores.headroom","info","Saturated. The best is 96%, past 85% since June 2025."],["import.importer","pass","We read the board on mlcommons.org: 10 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ailuminate-jailbreak","name":"MLCommons AILuminate Jailbreak","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","39 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on ailuminate.mlcommons.org: 39 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mlcr-aa","name":"MLCR-AA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mle-bench","name":"MLE-bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries), launch posts (3 scores) and system cards (12 scores)."],["scores.models","pass","12 models we track, plus 26 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 96.1%, past 85% since September 2026."],["import.importer","pass","We read the board on github.com: 27 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mle-bench-30","name":"MLE-bench 30","summary":{"pass":5,"warn":1,"fail":0,"info":4,"text":"5 of 6 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (8 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from June 2026, for Muse Spark."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 36.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mlgym-bench-meta","name":"MLGym-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 16 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mlogiqa","name":"MLogiQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries), launch posts (1 score) and system cards (8 scores)."],["scores.models","pass","25 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Apertus 70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 77.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 17 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mlqa","name":"MLQA","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mlrc-bench","name":"MLRC-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3-mini (#124)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mls-bench","name":"MLS-Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries), launch posts (5 scores) and system cards (1 score)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from September 2026, for Step 5 Preview."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 50.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mls-bench.com: 15 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mlvu","name":"MLVU","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (35 entries), launch posts (21 scores) and system cards (8 scores)."],["scores.models","pass","31 models we track, plus 26 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","info","Saturated. The best is 90.8%, past 85% since February 2026."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 26 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mm-alignbench","name":"MM-AlignBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","7 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for InternVL2-8B-MPO."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 82.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 7 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mm-browsecomp","name":"MM-BrowseComp","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries), launch posts (5 scores) and system cards (2 scores)."],["scores.models","pass","18 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 52%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 21 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mm-clawbench","name":"MM-ClawBench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for MiniMax-M2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 75.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on filecdn.minimax.chat: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mm-fineval","name":"MM-FinEval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","14 models we track, plus 5 other entries on its leaderboard."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 88%, past 85% since February 2026."],["import.importer","pass","We read the board on arxiv.org: 14 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mm-helix","name":"MM-HELIX","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","19 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for MiniCPM-V 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 84.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mm-helix.github.io: 38 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mm-ifeval","name":"MM-IFEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries), launch posts (3 scores) and system cards (2 scores)."],["scores.models","pass","13 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#167)."],["scores.headroom","pass","The best is 65.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mm-mt-bench","name":"MM-MT-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries), launch posts (4 scores) and system cards (5 scores)."],["scores.models","pass","15 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for STEP3-VL-10B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mm-niah","name":"MM-NIAH","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","2 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Idefics2-8B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 11.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mm-ocean","name":"MM-OCEAN","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mm-safetybench","name":"MM-SafetyBench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mm-toolsandbox","name":"MM-ToolSandbox","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mm-vet","name":"MM-Vet","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (278 entries), launch posts (8 scores) and system cards (7 scores)."],["scores.models","pass","171 models we track, plus 112 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Reka Edge."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 82.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on OpenCompass: 167 rows for models we track and 112 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmae","name":"MMAE","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmar","name":"MMAR","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (35 entries), launch posts (4 scores) and system cards (3 scores)."],["scores.models","pass","8 models we track, plus 32 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for NVIDIA Nemotron 3.5 Transcribe 30B-A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 32 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmarco","name":"mMARCO","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmart-bench","name":"MMArt-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","1 model we track, plus 7 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmau","name":"MMAU","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries), launch posts (12 scores) and system cards (13 scores)."],["scores.models","pass","16 models we track, plus 20 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Inkling-Small (#68)."],["scores.headroom","pass","The best is 82.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sakshi113.github.io: 1 row for models we track and 20 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmau-v05-15-25","name":"MMAU v05.15.25","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","7 models we track, plus 22 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Nova 2 Omni Preview."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 77.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sakshi113.github.io: 7 rows for models we track and 22 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmau-pro","name":"MMAU-Pro","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","14 models we track, plus 19 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.7 Flash (#26)."],["scores.headroom","pass","The best is 72.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 13 rows for models we track and 19 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmbench","name":"MMBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (226 entries), launch posts (6 scores) and system cards (11 scores)."],["scores.models","pass","162 models we track, plus 71 other entries on its leaderboard."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Cosmos3-Edge. That’s expected for a test replaced by MMBench-EN v1.1."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","info","Saturated. The best is 92%, past 85% since July 2024."],["import.importer","pass","We read the board on rank.opencompass.org.cn: 160 rows for models we track and 71 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 4 pairs differs by more than 10 points, the largest for Janus-Pro-7B (+17 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmbench-cn-v1-1","name":"MMBench-CN v1.1","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (280 entries), launch posts (3 scores) and system cards (5 scores)."],["scores.models","pass","183 models we track, plus 102 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Reka Edge. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 88.6%, past 85% since July 2024."],["import.importer","pass","We read the board on rank.opencompass.org.cn: 182 rows for models we track and 102 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmbench-en-v1-1","name":"MMBench-EN v1.1","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (280 entries), launch posts (22 scores) and system cards (11 scores)."],["scores.models","pass","199 models we track, plus 102 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for North Micro Vision Instruct."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 93.7%, past 85% since July 2024."],["import.importer","pass","We read the board on rank.opencompass.org.cn: 182 rows for models we track and 102 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmbench-gui","name":"MMBench-GUI","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for STEP3-VL-10B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 81.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmbench-video","name":"MMBench-Video","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (45 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","29 models we track, plus 17 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for InternVL3.5-241B-A28B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#167)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on rank.opencompass.org.cn: 31 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmcircuiteval","name":"MMCircuitEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","21 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Reka Edge."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#172)."],["scores.headroom","pass","The best is 69.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on OpenReward: 21 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmdm","name":"MMDM","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries)."],["scores.models","pass","36 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on mmdm-benchmark.github.io: 36 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmdocir","name":"MMDocIR","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmdu","name":"MMDU","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","5 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Idefics2-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on liuziyu77.github.io: 5 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mme","name":"MME","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (58 entries), launch posts (5 scores) and system cards (6 scores)."],["scores.models","pass","23 models we track, plus 41 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for LFM2.5-VL-3B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 17 rows for models we track and 41 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mme-cc","name":"MME-CC","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","14 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Seed1.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 57%, under 85%, so it still separates models."],["import.importer","pass","We read the board on randomtutu.github.io: 12 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mme-cot","name":"MME-CoT","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","10 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Kimi k1.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 64.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mmecot.github.io: 10 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mme-realworld","name":"MME-RealWorld","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries), launch posts (2 scores) and system cards (4 scores)."],["scores.models","pass","15 models we track, plus 25 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Nemotron Nano 12B v2 VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o mini (#178)."],["scores.headroom","pass","The best is 65.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mme-realworld.github.io: 11 rows for models we track and 25 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mme-reasoning","name":"MME-Reasoning","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","11 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Skywork-R1V3-38B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","pass","The best is 59.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on alpha-innovator.github.io: 13 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mme-videoocr","name":"MME-VideoOCR","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","15 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen3.5-Omni-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 77%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 14 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmeb","name":"MMEB","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (79 entries)."],["scores.models","pass","2 models we track, plus 77 other entries on its leaderboard."],["scores.recent","info","The newest came 10 months ago, in January 2026, for Qwen3-VL-Embedding-2B. That’s expected for a test replaced by MMEB-V2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 80.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 2 rows for models we track and 77 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmeb-v2","name":"MMEB-V2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","3 models we track, plus 43 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 77.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 2 rows for models we track and 43 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmeb-v3","name":"MMEB-V3","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (87 entries)."],["scores.models","pass","2 models we track, plus 85 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Qwen3-VL-Embedding-2B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 32.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 2 rows for models we track and 85 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmenterprise-bench","name":"MMEnterprise-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 68%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 20 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmgenre","name":"MMGenre","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmie","name":"MMIE","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmiu","name":"MMIU","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries) and system cards (1 score)."],["scores.models","pass","11 models we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for InternVL3.5-241B-A28B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 61.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mmiu-bench.github.io: 10 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmk12","name":"MMK12","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries)."],["scores.models","pass","10 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 73.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 10 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlongbench","name":"MMLongBench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (4 scores) and system cards (2 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 78.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlongbench-doc","name":"MMLongBench-Doc","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries), launch posts (18 scores) and system cards (10 scores)."],["scores.models","pass","28 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 66.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 17 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 8 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu","name":"MMLU","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries), launch posts (98 scores), system cards (90 scores) and claims Epoch AI collected (85 models)."],["scores.models","pass","192 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for NVIDIA Nemotron 3.5 Transcribe 30B-A4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V4-Pro (#70)."],["scores.headroom","info","Saturated. The best is 92.3%, past 85% since March 2023."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","warn","1 finding disputed or fixed only in a copy: Wrong labels and unclear tasks."]]},{"id":"mmlu-anatomy","name":"MMLU Anatomy","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Saturated. The best is 89%, past 85% since August 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-calibration-openai","name":"MMLU calibration (expected calibration error)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 3 years ago, in March 2023, for GPT-4. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-clinical-knowledge","name":"MMLU Clinical Knowledge","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Saturated. The best is 92%, past 85% since August 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-college-biology","name":"MMLU College Biology","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Saturated. The best is 95%, past 85% since August 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-college-medicine","name":"MMLU College Medicine","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 84%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-early-training","name":"MMLU Early Training","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-medical","name":"MMLU Medical","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (182 entries)."],["scores.models","pass","16 models we track, plus 166 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for MAmmoTH2-8B-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-3.5 Turbo (#201)."],["scores.headroom","pass","The best is 72.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Open Medical LLM Leaderboard: 20 rows for models we track and 166 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-medical-genetics","name":"MMLU Medical Genetics","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Saturated. The best is 96%, past 85% since August 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-professional-medicine","name":"MMLU Professional Medicine","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Saturated. The best is 94%, past 85% since August 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-azure-translated-openai","name":"MMLU translated with Azure Translate","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 3 years ago, in March 2023, for GPT-4. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-cf","name":"MMLU-CF","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (44 entries)."],["scores.models","pass","37 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for Phi-4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Phi-4 (#160)."],["scores.headroom","pass","The best is 73.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 74 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-pro","name":"MMLU-Pro","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (240 entries), launch posts (137 scores) and system cards (103 scores)."],["scores.models","pass","320 models we track, plus 28 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for NVIDIA Nemotron 3.5 Transcribe 30B-A4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 91.2%, past 85% since December 2024."],["import.importer","pass","We read the board on huggingface.co: 233 rows for models we track and 28 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","2 large gaps of 62 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-prox","name":"MMLU-ProX","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (36 entries), launch posts (21 scores) and system cards (8 scores)."],["scores.models","pass","52 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.7-Max (#49)."],["scores.headroom","info","Saturated. The best is 87%, past 85% since May 2026."],["import.importer","pass","We read the board on arxiv.org: 35 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-prox-lite","name":"MMLU-ProX Lite","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","10 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-235B-A22B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","pass","The best is 74.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 26 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-redux","name":"MMLU-Redux","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (32 scores) and system cards (25 scores)."],["scores.models","pass","45 models we track."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.7-Max (#49)."],["scores.headroom","info","Saturated. The best is 95%, past 85% since September 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-redux-2-0","name":"MMLU-Redux 2.0","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-redux-2-0-spanish","name":"MMLU-Redux 2.0 Spanish","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-sr","name":"MMLU-SR","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for GPT-4o mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o mini (#178)."],["scores.headroom","pass","The best is 65%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmlu-stem","name":"MMLU-stem","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","2 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Falcon-H1-34B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 83.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 11 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmmb","name":"MMMB","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from August 2026, for North Micro Vision Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 83%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmmlu","name":"MMMLU","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries), launch posts (56 scores) and system cards (35 scores)."],["scores.models","pass","75 models we track."],["scores.recent","pass","The newest is from August 2026, for K-EXAONE 2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.7 (#31)."],["scores.headroom","info","Saturated. The best is 92.7%, past 85% since December 2024."],["import.importer","pass","We read the board on github.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 5 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmmlu-lite-opencompass","name":"MMMLU-Lite","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmmu","name":"MMMU","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (186 entries), launch posts (93 scores) and system cards (51 scores)."],["scores.models","pass","202 models we track, plus 49 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","info","Saturated. The best is 86%, past 85% since November 2025."],["import.importer","pass","We read the board on mmmu-benchmark.github.io: 140 rows for models we track and 49 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 39 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmmu-pro","name":"MMMU-Pro","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (61 entries), launch posts (83 scores) and system cards (46 scores)."],["scores.models","pass","132 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 84.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mmmu-benchmark.github.io: 65 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 17 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmmu-pro-extended-mercor","name":"MMMU-Pro Extended","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","38 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 61.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mercor.com: 39 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmou","name":"MMOU","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and system cards (1 score)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 78.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmsearch","name":"MMSearch","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GLM-5V-Turbo."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 72.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmsearch-plus","name":"MMSearch-Plus","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries) and launch posts (2 scores)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 41.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmsi-bench","name":"MMSI-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries), launch posts (3 scores) and system cards (5 scores)."],["scores.models","pass","35 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 49.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 28 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmsi-video-bench","name":"MMSI-Video-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries) and system cards (1 score)."],["scores.models","pass","22 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 38%, under 85%, so it still separates models."],["import.importer","pass","We read the board on rbler1234.github.io: 35 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmstar","name":"MMStar","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (277 entries), launch posts (35 scores) and system cards (22 scores)."],["scores.models","pass","209 models we track, plus 103 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","info","Saturated. The best is 85.9%, past 85% since August 2026."],["import.importer","pass","We read the board on rank.opencompass.org.cn: 180 rows for models we track and 103 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmsu","name":"MMSU","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries), launch posts (4 scores) and system cards (4 scores)."],["scores.models","pass","11 models we track, plus 14 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 86.5%, past 85% since April 2026."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmt-bench","name":"MMT-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (33 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","18 models we track, plus 18 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 75.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 16 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmtu","name":"MMTU","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","10 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 69.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 10 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmvp","name":"MMVP","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (8 entries), launch posts (2 scores) and system cards (5 scores)."],["scores.models","pass","6 models we track, plus 7 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Seed1.8. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K2.5 (#76)."],["scores.headroom","info","Saturated. The best is 87%, past 85% since December 2025."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 7 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mmvu","name":"MMVU","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries), launch posts (22 scores) and system cards (14 scores)."],["scores.models","pass","54 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 82.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mmvu-benchmark.github.io: 28 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mobile-actions","name":"Mobile Actions","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Reka Edge. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 88.4%, past 85% since March 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mobile-eval","name":"Mobile-Eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mobile-eval-e","name":"Mobile-Eval-E","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mobile-eval-2","name":"Mobile-Eval-v2","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mobileagentbench","name":"MobileAgentBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in December 2023, for CogAgent."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 8.00%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mobileagentbench.github.io: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mobilegym","name":"MobileGym","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","5 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Qwen 3.6 Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 58.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mobilegym.dev: 5 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mobileminiwob","name":"MobileMiniWoB++","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Qwen2.5-VL-72B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","pass","The best is 68%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mobilepa-bench","name":"MobilePA-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","20 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 76.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tongyi-mai.github.io: 20 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mobileworld","name":"MobileWorld","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries), launch posts (3 scores) and system cards (1 score)."],["scores.models","pass","21 models we track, plus 24 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 77.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tongyi-mai.github.io: 20 rows for models we track and 24 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"model-slant","name":"Model Slant","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","warn","The record links a board on modelslant.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"model-spec-conversation-replay-openai","name":"Model Spec Conversation Replay","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"model-spec-evals-openai","name":"Model Spec Evals","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"model-spec-targeted-cases-openai","name":"Model Spec Targeted Cases","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 87.8%, past 85% since November 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"modelbench-reliability-under-constraints","name":"ModelBench: Reliability Under Constraints","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Sonnet 5.5 (#9)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moi-bench","name":"MoI-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","5 models we track, plus 14 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Flash Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mol-instructions","name":"Mol-Instructions","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Intern-S2-397B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 54%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mol-instructions-chemical-entity-recognition","name":"Mol-Instructions Chemical Entity Recognition","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on raw.githubusercontent.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mol-instructions-chemical-disease-interaction","name":"Mol-Instructions Chemical-disease Interaction Extraction","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on raw.githubusercontent.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mol-instructions-chemical-protein-interaction","name":"Mol-Instructions Chemical-protein Interaction Extraction","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on raw.githubusercontent.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mol-instructions-molecule-design","name":"Mol-Instructions Description-guided Molecule Design","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on raw.githubusercontent.com: 10 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mol-instructions-forward-reaction","name":"Mol-Instructions Forward Reaction Prediction","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on raw.githubusercontent.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mol-instructions-multiple-choice","name":"Mol-Instructions Multi-choice","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on raw.githubusercontent.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mol-instructions-reagent-prediction","name":"Mol-Instructions Reagent Prediction","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on raw.githubusercontent.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mol-instructions-retrosynthesis","name":"Mol-Instructions Retrosynthesis","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on raw.githubusercontent.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mol-instructions-true-false","name":"Mol-Instructions True or False","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on raw.githubusercontent.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"molecular-biology-capabilities-test-securebio","name":"Molecular Biology Capabilities Test","summary":{"pass":11,"warn":1,"fail":0,"info":0,"text":"11 of 12 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (104 entries) and system cards (9 scores)."],["scores.models","pass","104 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 68.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on securebio.org: 102 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moleculariq","name":"MolecularIQ","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries) and launch posts (1 score)."],["scores.models","pass","18 models we track, plus 17 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Intern-S2-397B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-R1 (#122)."],["scores.headroom","pass","The best is 62.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 21 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"molmoact2-sim-eval","name":"MolmoAct2 sim_eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"molmospaces","name":"MolmoSpaces","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","1 model we track, plus 33 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for MolmoAct 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 37.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on molmospaces.allen.ai: 33 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moltron-llm-showdown-1","name":"MOLTRON LLM Showdown v1.0","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. That’s expected for a test replaced by MOLTRON LLM Showdown v2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","pass","The best is 73%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ageofmoltron.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moltron-llm-showdown-2","name":"MOLTRON LLM Showdown v2.0","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. That’s expected for a test replaced by MOLTRON LLM Showdown v3.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","pass","The best is 55.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ageofmoltron.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moltron-llm-showdown-3","name":"MOLTRON LLM Showdown v3.0","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 69%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ageofmoltron.com: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moment-eval","name":"MOMENT Eval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries)."],["scores.models","pass","41 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 95.6%, past 85% since November 2025."],["import.importer","pass","We read the board on moment.corestudio.io: 68 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moment-finding-google","name":"Moment finding (Google DeepMind)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry) and launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Gemini Robotics ER 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 91.3%, past 85% since July 2026."],["import.importer","pass","We read the board on blog.google: 1 row for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moneda-bench-advicebench","name":"Moneda AdviceBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on benchmarks.finance: 1 entry at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moneda-bench","name":"Moneda Bench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on benchmarks.finance that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moneda-bench-citebench","name":"Moneda CiteBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on benchmarks.finance: 1 entry at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moneda-bench-filingsbench","name":"Moneda FilingsBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on benchmarks.finance: 1 entry at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moneda-bench-groundbench","name":"Moneda GroundBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on benchmarks.finance: 1 entry at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moneda-bench-portfoliobench","name":"Moneda PortfolioBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on benchmarks.finance: 1 entry at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"money-talks-google","name":"Money talks","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 2 years ago, in July 2024, for Gemma 2 27B. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemma 2 27B (#190)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompted-sandbagging-monitor-evasion-openai","name":"Monitor Evasion Capability Under Prompted Sandbagging","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"monitorability-sandbagging-openai","name":"Monitorability Sandbagging","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 61.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"monitorbench","name":"MonitorBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","info","The newest came 11 months ago, in December 2025, for Gemini 3 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","info","Saturated. The best is 91.9%, past 85% since April 2025."],["import.importer","pass","We read the board on proceedings.colm.cc: 72 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"monorepo-bench-openai","name":"Monorepo-Bench","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 76%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"monsoon-asr","name":"Monsoon ASR","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"monthly-swebench","name":"Monthly-SWEBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 76%, under 85%, so it still separates models."],["import.importer","pass","We read the board on unipat.ai: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moosebench","name":"MooseBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","6 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 31 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moral-reasoning-under-uncertainty","name":"Moral Reasoning under Uncertainty","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"moral-stories","name":"Moral Stories","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"morebench","name":"MoReBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash-Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 26 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"morebench-theory","name":"MoReBench-Theory","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash-Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 26 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"morphology-to-molecule-matching-anthropic","name":"Morphology-to-molecule matching (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 34%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"morse-500","name":"MORSE-500","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","13 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Seed2.0 Lite (0428)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 37.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on morse-500.github.io: 11 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-mortgagetax","name":"MortgageTax","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (88 entries)."],["scores.models","pass","87 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","pass","The best is 72.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 97 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mosaic-eval-gauntlet-original","name":"Mosaic Eval Gauntlet Original","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mosaic-eval-gauntlet-v0-3-0","name":"Mosaic Eval Gauntlet v0.3.0","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","6 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for Mistral-7B-Instruct-v0.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Mistral 7B v0.1 (#212)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 6 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mose","name":"MOSE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries)."],["scores.models","pass","17 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on henghuiding.github.io: 17 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mosev2","name":"MOSEv2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries)."],["scores.models","pass","20 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 20 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"motionbench","name":"MotionBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries), launch posts (8 scores) and system cards (8 scores)."],["scores.models","pass","20 models we track, plus 11 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 75.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on motion-bench.github.io: 13 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"motoverse","name":"MotoVerse","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","4 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Qwen3-VL-2B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 46.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"movie-gen-video-bench","name":"Movie Gen Video Bench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mp-docvqa","name":"MP-DocVQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on raw.githubusercontent.com: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mpdocbench-parse","name":"MPDocBench-Parse","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Unlimited OCR 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 84.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mpie-bench","name":"MPIE-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","5 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for BAGEL-7B-MoT."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mquake","name":"MQuAKE","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mr-niah-minimax","name":"MR-NIAH (Multi-Round Needles-In-A-Haystack)","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for MiniMax-Text-01."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mrcr-v1","name":"MRCR (Michelangelo, original)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (6 scores) and system cards (7 scores)."],["scores.models","pass","9 models we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Gemini 2.5 Flash. That’s expected for a test replaced by MRCR v2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.5 Pro (#104)."],["scores.headroom","info","Saturated. The best is 94.5%, past 85% since March 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mrcr-v2","name":"MRCR v2","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (29 scores) and system cards (18 scores)."],["scores.models","pass","30 models we track."],["scores.recent","pass","The newest is from October 2026, for Mercury 3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.7 Flash (#26)."],["scores.headroom","info","Saturated. The best is 98.5%, past 85% since May 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mri-dataset-1-google","name":"MRI Dataset 1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mrobust04","name":"mRobust04","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ms-marco-qa","name":"MS MARCO Question Answering","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (65 entries)."],["scores.models","pass","65 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on microsoft.github.io: 65 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ms-cxr-t","name":"MS-CXR-T","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on openaccess.thecvf.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"msd-i-multimodal-genre-classification","name":"MSD-I Multimodal Genre Classification","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on transactions.ismir.net: 9 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"msearth","name":"MSEarth","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (23 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","20 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Intern-S1-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 68.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 29 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mseb","name":"MSEB","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","10 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 14 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"msqa-bytedance","name":"MSQA","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 50.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"msr-vtt","name":"MSR-VTT","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"msr-vtt-retrieval","name":"MSR-VTT retrieval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini Embedding 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"msrvtt-qa","name":"MSRVTT-QA","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"msvd","name":"MSVD","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ms2-medical-studies","name":"MS²","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mt-aime-cohere-internal","name":"MT-AIME (Cohere internal)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Command A+. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 86%, past 85% since May 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mt-aime2024","name":"MT-AIME2024","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries), launch posts (1 score) and system cards (7 scores)."],["scores.models","pass","11 models we track."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Qwen3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3-mini (#124)."],["scores.headroom","pass","The best is 80.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 4 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mt-bench","name":"MT-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries), launch posts (14 scores) and system cards (16 scores)."],["scores.models","pass","28 models we track, plus 29 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Granite 4.1 30B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 29 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mt-bench-101","name":"MT-Bench-101","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries) and launch posts (1 score)."],["scores.models","pass","7 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for InternLM3-8B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen-14B (#214)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mteb","name":"MTEB","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (145 entries), launch posts (4 scores) and system cards (2 scores)."],["scores.models","pass","4 models we track, plus 141 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 72.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mteb-leaderboard-backend.hf.space: 4 rows for models we track and 141 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mteb-arena","name":"MTEB Arena","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 14 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mteb-br","name":"MTEB-BR","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (93 entries)."],["scores.models","pass","4 models we track, plus 89 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Gemini Embedding 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 4 rows for models we track and 89 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mtg-jamendo","name":"MTG-Jamendo","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mtgbench","name":"MTGBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mth-v2","name":"MTHv2","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (31 entries)."],["scores.models","pass","11 models we track, plus 20 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 86.4%, past 85% since April 2026."],["import.importer","pass","We read the board on TongGuOCR: 11 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mtob","name":"MTOB","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores) and system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 4 Maverick (#153)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mtop","name":"MTOP","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mts-dialog","name":"MTS-Dialog","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mtsamples-clinical-entity-recognition","name":"MTSamples clinical entity recognition","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on pmc.ncbi.nlm.nih.gov: 2 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mtvqa-test","name":"MTVQA-Test","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries), launch posts (2 scores) and system cards (5 scores)."],["scores.models","pass","24 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 56.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 18 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mu-bench","name":"MU-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","3 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.5 Transcribe."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on research.sierra.ai: 4 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"muchomusic","name":"MuChoMusic","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries), launch posts (4 scores) and system cards (3 scores)."],["scores.models","pass","4 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 72.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mucs-2021","name":"MUCS 2021","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"muirbench","name":"MuirBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries), launch posts (9 scores) and system cards (12 scores)."],["scores.models","pass","21 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 81.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mulset","name":"MulSeT","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Qwen2.5-VL-7B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 36.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 35 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mac-bench","name":"Multi Agent Coordination Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on benchmarks.bespokelabs.ai: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"olam-arena-codenames","name":"Multi-Agent Arena Codenames","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on olamlabs.ai: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"olam-arena-diplomacy","name":"Multi-Agent Arena Diplomacy","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 37.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on olamlabs.ai: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"olam-arena-risk","name":"Multi-Agent Arena Risk","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 38.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on olamlabs.ai: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"olam-arena-social-poker","name":"Multi-Agent Arena Social Poker","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","34 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on olamlabs.ai: 34 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multi-agent-search-anthropic","name":"Multi-agent search (Anthropic internal benchmark)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Claude Opus 4.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","pass","The best is 74.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multi-crit","name":"Multi-Crit","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (25 entries)."],["scores.models","pass","17 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Qwen3-VL-8B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 43%, under 85%, so it still separates models."],["import.importer","pass","We read the board on multi-crit.github.io: 17 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multi-dialect-sql-cohere","name":"Multi-dialect SQL evaluation (Cohere)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Cohere Command A. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 55.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multi-factual-anthropic","name":"Multi-factual (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 2 years ago, in March 2024, for Claude 3 Opus. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 62.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multi-hop-search-factuality-google","name":"Multi-hop search factuality","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 94.3%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multi-if","name":"Multi-IF","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (20 scores) and system cards (15 scores)."],["scores.models","pass","28 models we track."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3-235B-A22B-Thinking (#108)."],["scores.headroom","info","Saturated. The best is 89.1%, past 85% since February 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multi-lcb","name":"Multi-LCB","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (22 entries)."],["scores.models","pass","16 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for gpt-oss-120b."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-235B-A22B-Thinking (#108)."],["scores.headroom","pass","The best is 67.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 18 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multi-lexsum","name":"Multi-LexSum","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"msad","name":"Multi-Scenario Anomaly Detection","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on msad-dataset.github.io: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multi-swe-bench","name":"Multi-SWE-bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries), launch posts (9 scores) and system cards (6 scores)."],["scores.models","pass","10 models we track, plus 34 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for MiniMax-M2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K2 Thinking (#94)."],["scores.headroom","pass","The best is 52.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on multi-swe-bench.github.io: 33 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multi-swe-bench-flash","name":"Multi-SWE-bench flash","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries) and launch posts (1 score)."],["scores.models","pass","6 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Qwen3-Coder-480B-A35B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 27%, under 85%, so it still separates models."],["import.importer","pass","We read the board on multi-swe-bench.github.io: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multi-swe-bench-mini","name":"Multi-SWE-bench mini","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","8 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Qwen3-Coder-480B-A35B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 25.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 19 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scale-multi-turn-human-jailbreaks","name":"Multi-Turn Human Jailbreaks","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sudoku-m8ngotree","name":"Multi-Turn Sudoku (m8ngotree)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multi-turn-testing-anthropic","name":"Multi-turn testing (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multiblimp","name":"MultiBLiMP","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries)."],["scores.models","pass","25 models we track, plus 15 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for OLMo 2 1B Instruct. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3-14B (#139)."],["scores.headroom","info","Saturated. The best is 90.2%, past 85% since April 2024."],["import.importer","pass","We read the board on github.com: 27 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multicardioner","name":"MultiCardioNER","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on participants-area.bioasq.org that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multichallenge","name":"MultiChallenge","summary":{"pass":11,"warn":1,"fail":0,"info":0,"text":"11 of 12 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (47 entries), launch posts (36 scores) and system cards (21 scores)."],["scores.models","pass","80 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 82%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 46 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 7 pairs differs by more than 10 points, the largest for Qwen3.5-4B (+24 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multichallenge-audio-openai","name":"MultiChallenge Audio (OpenAI)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for gpt-realtime. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 30.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multichallenge-audio-xiaomi","name":"MultiChallenge Audio (Xiaomi)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for MiMo-Audio-7B-Instruct. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 15.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multiclinai","name":"MultiClinAI","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multiclinsum","name":"MultiClinSum","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in September 2024, for Qwen2.5-0.5B. That’s expected for a test replaced by MultiClinSum-2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ceur-ws.org: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multiclinsum-2","name":"MultiClinSum-2","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multicochrane","name":"MultiCochrane","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multieurlex","name":"MultiEURLEX","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (178 entries)."],["scores.models","pass","4 models we track, plus 174 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MTEB: 4 rows for models we track and 174 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multifinben","name":"MultiFinBen","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","9 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Llama 4 Scout."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3-mini (#124)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 9 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multifinground","name":"MultiFinGround","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","9 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Saturated. The best is 95.4%, past 85% since October 2025."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multigec-2025","name":"MultiGEC 2025","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 2 9B (#202)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on spraakbanken.github.io: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multihop-rag","name":"MultiHop-RAG","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for Claude 2.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","pass","The best is 56%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multilingual-aime-2024-mistral","name":"Multilingual AIME 2024 (Mistral)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Magistral Medium 1.0. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multilingual-conversational-speech-language-model-challenge","name":"Multilingual Conversational Speech Language Model Challenge","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"melo-occupations","name":"Multilingual Entity Linking of Occupations","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 12 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mels-skill-linking","name":"Multilingual Entity Linking of Skills","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multilingual-exams","name":"Multilingual Exams","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries) and launch posts (3 scores)."],["scores.models","pass","3 models we track, plus 36 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Qwen2.5-72B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 78.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on qwenlm.github.io: 36 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multilingual-hatecheck","name":"Multilingual HateCheck","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multilingual-human-evaluation","name":"Multilingual human evaluation","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Qwen2-72B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on qwenlm.github.io: 6 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multilingual-ifeval","name":"Multilingual IFEval","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for Qwen2.5-14B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","info","Saturated. The best is 87%, past 85% since September 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multilingual-librispeech","name":"Multilingual LibriSpeech","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Nova 2 Omni Preview."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multilingual-mmbench","name":"Multilingual MMBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Nemotron Nano 12B v2 VL."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multilingual-mmlu-meta","name":"Multilingual MMLU (Meta)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 2 years ago, in November 2024, for Llama 3.1-405B. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","pass","The best is 83.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multilingual-expert-domains-openai","name":"Multilingual performance on translated expert-domain tasks","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 16 months ago, in July 2025, for ChatGPT agent. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multilingual-safety-google","name":"Multilingual Safety","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multilingual-understanding","name":"Multilingual Understanding","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 2 years ago, in September 2024, for Qwen2.5-72B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","info","Saturated. The best is 89.6%, past 85% since September 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multiloko","name":"MultiLoKo","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","pass","The best is 51.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 11 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multimedia-terminalbench","name":"MultiMedia-TerminalBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (21 entries)."],["scores.models","pass","5 models we track, plus 16 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-122B-A10B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 16.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 24 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multimedqa","name":"MultiMedQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 4 years ago, in April 2022, for PaLM (540B)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 58.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multimodal-policy-red-teaming-anthropic","name":"Multimodal policy red-teaming (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 2 years ago, in March 2024, for Claude 3 Opus. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Saturated. The best is 99.2%, past 85% since March 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multimodal-rewardbench","name":"Multimodal RewardBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in September 2024, for Aria. That’s expected for a test replaced by Multimodal RewardBench 2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.2 90B (#179)."],["scores.headroom","pass","The best is 62.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multimodal-rewardbench-2","name":"Multimodal RewardBench 2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","12 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 76.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 12 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multimodal-troubleshooting-virology-openai","name":"Multimodal Troubleshooting Virology","summary":{"pass":7,"warn":0,"fail":0,"info":3,"text":"All 7 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (34 scores)."],["scores.models","pass","29 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 63.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multinerd","name":"MultiNERD","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multinrc","name":"MultiNRC","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (38 entries)."],["scores.models","pass","38 models we track."],["scores.recent","pass","The newest is from July 2026, for Muse Spark 1.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Pro (#19)."],["scores.headroom","pass","The best is 65.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 44 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multipanelvqa","name":"MultipanelVQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for Gemini 1.0 Pro Vision."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 72.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 5 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multipl-e","name":"MultiPL-E","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (60 entries), launch posts (11 scores) and system cards (13 scores)."],["scores.models","pass","27 models we track, plus 51 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Granite 4.1 30B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K2 (#117)."],["scores.headroom","info","Saturated. The best is 88.2%, past 85% since July 2025."],["import.importer","pass","We read the board on huggingface.co: 9 rows for models we track and 51 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multirc","name":"MultiRC","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries) and launch posts (1 score)."],["scores.models","pass","4 models we track, plus 22 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in May 2023, for PaLM 2-L. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is PaLM 2-L (#198)."],["scores.headroom","info","Saturated. The best is 88.7%, past 85% since October 2019."],["import.importer","pass","We read the board on super.gluebenchmark.com: 3 rows for models we track and 22 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multiref-compass","name":"MultiRef-Compass","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multiturn-jailbreak-evals-openai","name":"Multiturn Jailbreak Evaluations","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 80.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mumin","name":"MuMiN","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on mumin-dataset.github.io: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"musdb-alt","name":"MUSDB-ALT","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on futuremirex.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"musdb18","name":"MUSDB18","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"muse","name":"MUSE","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (30 entries)."],["scores.models","pass","29 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 56.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 29 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"muse-cad-assemblies","name":"MUSE (CAD assemblies)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 52.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on dong7313.github.io: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"muse-targeted-cyber-misuse-meta","name":"Muse-targeted cyber misuse and violation red-teaming","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"musebench","name":"MUSEBENCH","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"musebench-zhao-music-understanding","name":"MuseBench (Music Score and Performance Understanding)","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Qwen2.5-Omni-7B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 52.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"music-arena","name":"Music Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","3 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Stable Audio 3 Medium."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on music-arena.org: 3 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"music-arena-vocal","name":"Music Arena Vocal","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","2 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Lyria 3 Clip."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on music-arena.org: 2 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"music-instrument-separation","name":"MUSIC Instrument Separation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on openaccess.thecvf.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"music-understanding-benchmark","name":"Music Understanding Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"music-21-instrument-separation","name":"MUSIC-21 Instrument Separation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"musica-metabench","name":"MusICA-MetaBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","4 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Gemini 3.1 Flash-Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 59%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"musical-expectation-benchmark","name":"Musical Expectation Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"musiceval","name":"MusicEval","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on sites.google.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"musictheorybench","name":"MusicTheoryBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","8 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 71.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MuTheoryEval: 10 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"musixqa","name":"MusiXQA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for Phi-3-vision-128k-instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 12.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"musr","name":"MuSR","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (244 entries), launch posts (6 scores) and system cards (2 scores)."],["scores.models","pass","106 models we track, plus 144 other entries on its leaderboard."],["scores.recent","info","The newest came 11 months ago, in December 2025, for Hermes 4.3 36B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is QwQ-32B (#137)."],["scores.headroom","info","Saturated. The best is 86.7%, past 85% since September 2025."],["import.importer","pass","We read the board on Hugging Face: 106 rows for models we track and 144 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mustard","name":"MUStARD","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mustbench","name":"MusTBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","5 models we track, plus 11 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 50.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on daeyongkwon98.github.io: 4 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mutox","name":"MuTox","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mutual","name":"MuTual","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mvbench","name":"MVBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries), launch posts (18 scores) and system cards (12 scores)."],["scores.models","pass","28 models we track, plus 31 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GLM-5.3-Flash (#53)."],["scores.headroom","pass","The best is 79%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 5 rows for models we track and 31 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mvdrivebench","name":"MVDriveBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (23 entries)."],["scores.models","pass","15 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 80.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 30 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mvpbench","name":"MVPBench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (3 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Cosmos3-Edge. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 72.5%, under 85%, so it still separates models."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mwahaha","name":"MWAHAHA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on pln-fing-udelar.github.io: 7 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mycareersfuture-skill-retrieval","name":"MyCareersFuture Skill Retrieval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"myclaw-bench","name":"MyClaw Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mypcbench","name":"MyPCBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","9 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 62%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mypcbench.com: 9 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mystery-game-puzzles","name":"Mystery Game Puzzles","summary":{"pass":7,"warn":1,"fail":0,"info":2,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from Epoch AI’s runs (86 models)."],["scores.models","pass","86 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 95%, past 85% since November 2026."],["import.importer","warn","The record links a board on epoch.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-mysterymechanism","name":"MysteryMechanism","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries) and system cards (1 score)."],["scores.models","pass","35 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","8 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 53.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 34 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mythos-personalized-story-generation","name":"Mythos","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 1 entry at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"m3av-speech-recognition","name":"M³AV Speech Recognition","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on aclanthology.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"malromur","name":"Málrómur","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"n1-contingency-dispatch","name":"N-1 Contingency Dispatch","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"n2c2-2018-ade-medication-extraction","name":"n2c2 2018 Adverse Drug Events and Medication Extraction","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"n2c2-2022-contextualized-medication-event-extraction","name":"n2c2 2022 Contextualized Medication Event Extraction","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"n2c2-2022-assessment-plan-reasoning","name":"n2c2 2022 Progress Note Understanding: Assessment and Plan Reasoning","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nafd-e1-financial-reporting","name":"NAFD E1 Financial Reporting","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"naijaproverb-bench","name":"NaijaProverb-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","18 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 47.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 18 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"naijarc","name":"NaijaRC","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"naijasenti","name":"NaijaSenti","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","1 model we track, plus 19 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Qwen3-Embedding-8B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 48.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MTEB: 1 row for models we track and 19 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nakdimon-modern-hebrew-test","name":"Nakdimon Modern Hebrew Test Set","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nanogpt-expenditure-horizon-metr","name":"NanoGPT Expenditure Horizon","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nanogpt-speedrun","name":"NanoGPT Speedrun","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 42.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nanogpt-speedrun-frontier-prime-intellect","name":"NanoGPT Speedrun Frontier","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","18 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on primeintellect.ai: 109 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nanogpt-bench-intology","name":"NanoGPT-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 9.30%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"napisul-ii-progress-test-2023","name":"NAPISUL-II Progress Test 2023","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on scielo.br: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"narra-gym","name":"NARRA-Gym","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on narra-gym.github.io: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"narrativebench","name":"NarrativeBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is Gemini 4 Carbon (#3)."],["scores.headroom","info","Saturated. The best is 90.7%, past 85% since September 2026."],["import.importer","pass","We read the board on isle-dev.github.io: 21 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"narrativeqa","name":"NarrativeQA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (91 entries)."],["scores.models","pass","71 models we track, plus 20 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Gemini 2.0 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on HELM Lite: 71 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"narrativetrack","name":"NarrativeTrack","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"narrlv","name":"NarrLV","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","1 model we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Wan2.1-T2V-14B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on amap-ml.github.io: 1 row for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nasa-sde-ir-v5","name":"NASA SDE IR Benchmark v5","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nasa-ir","name":"NASA-IR","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nasa-qa","name":"NASA-QA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"native-code-exploitation-meta","name":"Native Code Exploitation (Meta)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 13 months ago, in October 2025, for Code World Model (CWM). Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 8.30%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nativelongbench","name":"NativeLongBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen3.8 Flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"natural-adversarial-objects","name":"Natural Adversarial Objects","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on labs.scale.com: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"natural-plan","name":"NATURAL PLAN","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"natural-questions","name":"Natural Questions","summary":{"pass":5,"warn":1,"fail":0,"info":4,"text":"5 of 6 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (8 scores) and system cards (17 scores)."],["scores.models","pass","20 models we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for InternVL3.5-241B-A28B. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","pass","The best is 53.4%, under 85%, so it still separates models."],["import.importer","warn","The record links a board on ai.google.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"natural-questions-in-icelandic","name":"Natural Questions in Icelandic","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"newt-2021","name":"Natural World Tasks (NeWT 2021)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (59 entries)."],["scores.models","pass","59 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on BioBench: 59 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"natural2code","name":"Natural2Code","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (5 scores) and system cards (5 scores)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from July 2026, for DiffusionGemma 26B A4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","info","Saturated. The best is 94%, past 85% since December 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"naturalbench","name":"NaturalBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (55 entries)."],["scores.models","pass","7 models we track, plus 48 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for o3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 55.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on linzhiqiu.github.io: 7 rows for models we track and 48 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"naturalcodebench","name":"NaturalCodeBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (39 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","9 models we track, plus 32 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for Seed Diffusion Preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 52.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track and 32 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"naturalproofs","name":"NaturalProofs","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"naturebench","name":"NatureBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","6 models we track, plus 18 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 23.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on frontisai.github.io: 24 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nautdata-test","name":"NautData test set","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","6 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 21 months ago, in February 2025, for Gemini 2.0 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","info","Saturated. The best is 91.9%, past 85% since October 2023."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"navi","name":"NAVI","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"navital-gym","name":"Navital Gym","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2026."],["import.importer","pass","We read the board on navitallabs.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nb-tale-module-1","name":"NB Tale Module 1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nb-tale-module-3","name":"NB Tale Module 3","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nbabench-damiba","name":"NBABench (Thierry Damiba)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nchlt-isizulu-speech","name":"NCHLT isiZulu Speech Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ncp-bench","name":"NCP-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Kimi K2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"needle-in-a-haystack","name":"Needle In A Haystack","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (8 scores)."],["scores.models","pass","9 models we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for MiniMax-Text-01. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since January 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"needlebench","name":"NeedleBench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in July 2024, for Qwen2-72B. That’s expected for a test replaced by NeedleBench V2."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"needlebench-v2","name":"NeedleBench V2","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (29 entries)."],["scores.models","pass","23 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for o4-mini. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o4-mini (#95)."],["scores.headroom","info","Saturated. The best is 86.3%, past 85% since March 2025."],["import.importer","pass","We read the board on arxiv.org: 27 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nego-eval","name":"Nego Eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nejumi-llm-leaderboard","name":"Nejumi LLM Leaderboard","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on wandb.ai: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nejumi-llm-leaderboard-2","name":"Nejumi LLM Leaderboard 2","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nejumi-llm-leaderboard-3","name":"Nejumi LLM Leaderboard 3","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (95 entries)."],["scores.models","pass","58 models we track, plus 37 other entries on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for gpt-oss-120b. That’s expected for a test replaced by Nejumi LLM Leaderboard 4."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on wandb.ai: 61 rows for models we track and 37 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nejumi-llm-leaderboard-4","name":"Nejumi LLM Leaderboard 4","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on wandb.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nepabench","name":"NEPABench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nepaquad","name":"NEPAQuAD","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for Mistral-7B-Instruct-v0.1."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 75.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"neptts-bench","name":"NepTTS-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","1 model we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Gemini 2.5 Flash TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"neptune","name":"Neptune","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries) and system cards (2 scores)."],["scores.models","pass","7 models we track, plus 8 other entries on its leaderboard."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Saturated. The best is 87.3%, past 85% since July 2025."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nerel-bench","name":"NEREL-bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nest","name":"NEST","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","12 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Gemini 3.1 Flash-Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 mini (#92)."],["scores.headroom","pass","The best is 76%, under 85%, so it still separates models."],["import.importer","pass","We read the board on laubravo.github.io: 12 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"network-ctf-anthropic","name":"Network CTF challenges (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since November 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"neural-activation-reading-for-collusion-benchmark","name":"Neural Activation Reading for Collusion Benchmark","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for gpt-oss-20b."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"neuralcad-edit","name":"neuralCAD-Edit","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on autodeskailab.github.io: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-neurips-2025","name":"NeurIPS 2025 Experts Benchmark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 73.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"neuro-reasoning","name":"Neuro Reasoning (dwin2)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"neurodiscoverybench","name":"NeuroDiscoveryBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"neurosciencebench-latchbio","name":"NeuroscienceBench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","22 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 41%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 36 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"neutrality-eval-google","name":"Neutrality evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 93%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nevir","name":"NevIR","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (21 entries)."],["scores.models","pass","21 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 21 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nevis-22","name":"NEVIS’22","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"new-synonym-benchmark-markovic","name":"New Synonym Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","13 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Granite 4.2 30B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 27B (#59)."],["scores.headroom","info","Saturated. The best is 99.3%, past 85% since December 2024."],["import.importer","pass","We read the board on markovic.com: 13 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"new-yorker-caption-contest-humor-understanding","name":"New Yorker Caption Contest Humor Understanding","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","2 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 years ago, in October 2019, for T5-11B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 70.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"news-with-gpt-instructions","name":"News With GPT Instructions","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Llama 2-13B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-13B (#223)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 4 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"newsbench-chinese-journalism","name":"NewsBench (Chinese journalism)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","6 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for GPT-4 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on iaar-shanghai.github.io: 6 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"newsbench-forum-ai","name":"NewsBench (Forum AI)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 88.3%, past 85% since July 2026."],["import.importer","pass","We read the board on byforum.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"newsguard-false-audio-claims","name":"NewsGuard false audio claims audit","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on newsguardtech.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"newsqa","name":"NewsQA","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"next-qa","name":"NExT-QA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","1 model we track, plus 7 other entries on its leaderboard."],["scores.recent","info","The newest came 13 months ago, in October 2025, for LongCat-Flash-Omni. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 86.2%, past 85% since October 2025."],["import.importer","pass","We read the board on github.com: 7 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nextjs-agent-evals","name":"Next.js Agent Evals","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","14 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since July 2026."],["import.importer","pass","We read the board on nextjs.org: 14 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nexus","name":"Nexus","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries), launch posts (1 score) and system cards (6 scores)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","pass","The best is 58.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nfcorpus","name":"NFCorpus","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (267 entries)."],["scores.models","pass","4 models we track, plus 263 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Qwen3-Embedding-0.6B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mteb-leaderboard-backend.hf.space: 4 rows for models we track and 263 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nia-sovereign-ai-phase-1","name":"NIA Sovereign AI Benchmark Evaluation, Phase 1","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nia-sovereign-ai-phase-2","name":"NIA Sovereign AI Benchmark Evaluation, Phase 2","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nicta-piboso","name":"NICTA-PIBOSO","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nifty-market-movement","name":"NIFTY Market Movement","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nigerian-pidgin-asr","name":"Nigerian Pidgin ASR","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on amina-mardiyyah.github.io: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"niilc-question-answering-dataset","name":"NIILC Question Answering Dataset","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nim-compiler-repair","name":"Nim compiler repair benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nim-hard","name":"Nim Hard","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nitibench","name":"NitiBench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Gemini 1.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nl2repo-bench","name":"NL2Repo-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries), launch posts (28 scores) and system cards (6 scores)."],["scores.models","pass","38 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for LongCat-2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek V4.1 Flash 1017 (#23)."],["scores.headroom","pass","The best is 68.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nlebench","name":"NLEBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nlpcc-essay-coherence-2023","name":"NLPCC 2023 Chinese Essay Discourse Coherence Evaluation","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on github.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nlpcc-essay-coherence-2024","name":"NLPCC 2024 Chinese Essay Discourse Logic Evaluation and Integration","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nlvr2","name":"NLVR2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on lil.nlp.cornell.edu: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"no-cot-math-time-horizon-uk-aisi","name":"No-CoT math time horizon","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as minutes, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"noadivines-safe-order-taking-in-informal-spanish","name":"NoAdivines: Safe Order-Taking in Informal Spanish","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nocaps","name":"nocaps","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on nocaps.org: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nocot-bench-ncri","name":"nocot-bench (NCRI)","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (287 entries) and system cards (2 scores)."],["scores.models","pass","202 models we track, plus 86 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","8 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 253 rows for models we track and 86 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nodal-pricing-lmp","name":"Nodal Pricing LMP","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"noisybench","name":"NoisyBench","summary":{"pass":7,"warn":3,"fail":0,"info":2,"text":"7 of 10 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#112)."],["scores.headroom","pass","The best is 72.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 20 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nolima","name":"NoLiMa","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","22 models we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Gemini 2.5 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.5 Flash (#114)."],["scores.headroom","info","Saturated. The best is 89.1%, past 85% since November 2024."],["import.importer","pass","We read the board on github.com: 22 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nolima-hard","name":"NoLiMa-Hard","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 78.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nollysenti","name":"NollySenti","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nocasa-2025","name":"Non-native Children's Automatic Speech Assessment Challenge","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on codabench.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"noor-mmlu-var","name":"Noor","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"noreval","name":"NorEval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (97 entries)."],["scores.models","pass","19 models we track, plus 78 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash-Base."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ltgoslo.github.io: 57 rows for models we track and 78 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"normad-eti","name":"NormAd-Eti","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","3 models we track, plus 17 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for OLMo-7B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-3.5 Turbo (#200)."],["scores.headroom","pass","The best is 81.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 11 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"norquad","name":"NorQuAD","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"norregel","name":"norregel","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","6 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 45.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 6 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"norsid","name":"NorSID","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"norsumm","name":"NorSumm","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","2 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for Mistral 7B v0.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Mistral 7B v0.1 (#212)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"norwegian-comma-benchmark","name":"Norwegian Comma Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"norwegian-courts-bitext-mining","name":"Norwegian Courts Bitext Mining","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (219 entries)."],["scores.models","pass","6 models we track, plus 213 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 100%, past 85% since January 2025."],["import.importer","pass","We read the board on MTEB: 6 rows for models we track and 213 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"norwegian-parliamentary-speech-corpus","name":"Norwegian Parliamentary Speech Corpus","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"norwegian-review-corpus","name":"Norwegian Review Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nota-music-notation","name":"NOTA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries)."],["scores.models","pass","10 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for MiniCPM-Llama3-V 2.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 17.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 10 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"noticia","name":"NoticIA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (36 entries)."],["scores.models","pass","11 models we track, plus 25 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Qwen1.5-14B-Chat."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-3.5 Turbo (#200)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 11 rows for models we track and 25 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"notsofar-1","name":"NOTSOFAR-1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"notsotiny-25-12","name":"NotSoTiny 25-12","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","9 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 31B IT."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K2.5 (#76)."],["scores.headroom","pass","The best is 31.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 9 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nova-63","name":"NOVA-63","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (60 entries) and launch posts (11 scores)."],["scores.models","pass","63 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.7-Max (#49)."],["scores.headroom","pass","The best is 61.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 54 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"novel-compiler-task-anthropic","name":"Novel compiler task (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 80.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"novel-logic-problems-expert-eval-zai","name":"Novel logical reasoning problems expert evaluation (Z.ai)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GLM-4.5-Air. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"novelty-judge-bench-ai2","name":"Novelty Judge Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"noveltybench","name":"NoveltyBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","27 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on novelty-bench.github.io: 35 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"npcbench","name":"NPCBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (21 entries)."],["scores.models","pass","13 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen3.6 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 23%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 13 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nphardeval","name":"NPHardEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","9 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for GPT-4 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#172)."],["scores.headroom","pass","The best is 37.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 9 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ntcir-13-medweb","name":"NTCIR-13 MedWeb","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ntcir-17-rr-tnm","name":"NTCIR-17 Radiology Report TNM Staging","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ntcir-17-sm-ade","name":"NTCIR-17 Social Media Adverse Drug Event Detection","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ntcir-18-hidden-rad","name":"NTCIR-18 HIDDEN-RAD","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on research.nii.ac.jp: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ntcir-18-mednlp-chat","name":"NTCIR-18 MedNLP-CHAT","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on sociocom.naist.jp: 6 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ntcir-19-mednlp-call-triage","name":"NTCIR-19 MedNLP-CALL Triage Classification","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ntrex-128","name":"NTREX-128","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Cohere Command A."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ndm-bench","name":"Nuclear Decision-Making Benchmark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 81.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nufo","name":"Nufo","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"numb3rs","name":"Numb3rs","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"numeric-grounding","name":"Numeric Grounding","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for GLM-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GLM-5 (#89)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"llm-meta-analysis","name":"Numerical results extraction from randomized controlled trials","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 77.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on 67AI Lab: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nusawrites","name":"NusaWrites","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","19 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 19 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nusax","name":"NusaX","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nuscene","name":"Nuscene","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (8 scores)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Qwen3.5-9B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5 397B-A17B (#78)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nuscenes-detection","name":"nuScenes Detection","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (335 entries)."],["scores.models","pass","335 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on nuscenes.org: 335 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nv-chat-benchmark-nvidia","name":"NV Chat Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nwpu-captions","name":"NWPU-Captions","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nwpu-resisc45","name":"NWPU-RESISC45","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (57 entries)."],["scores.models","pass","1 model we track, plus 56 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 88.3%, past 85% since October 2026."],["import.importer","pass","We read the board on MTEB: 1 row for models we track and 56 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nyt-connections-prime","name":"NYT Connections (Prime Community)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nyu-ctf-bench","name":"NYU CTF Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for GPT-4.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 22%, under 85%, so it still separates models."],["import.importer","pass","We read the board on nyu-llm-ctf.github.io: 12 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nyu-depth-v2","name":"NYU Depth V2","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"o-net-m6","name":"O-NET M6","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on openthaigpt.aieat.or.th that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"o11y-bench","name":"o11y-bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","22 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 93.8%, past 85% since September 2026."],["import.importer","pass","We read the board on o11ybench.ai: 69 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oab-exams","name":"OAB Exams","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oab-bench","name":"OAB-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Gemini 3.1 Flash-Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 16 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oai-repo-sabotage-v2-openai","name":"OAI-repo Sabotage v2","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oasis-agent-safety","name":"OASIS (agent safety)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 95.2%, past 85% since July 2025."],["import.importer","pass","We read the board on arxiv.org: 28 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"obi-bench","name":"OBI-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (23 entries)."],["scores.models","pass","15 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for mPLUG-Owl3-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#167)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 15 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"hf-object-detection-leaderboard","name":"Object Detection Leaderboard","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries)."],["scores.models","pass","30 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 30 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"objectnet","name":"ObjectNet","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"objectron","name":"Objectron","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Qwen3-VL-235B-A22B. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"obliq-bench","name":"OBLIQ-Bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Gemini Embedding 2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ocbench","name":"OCBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"occluded-video-instance-segmentation","name":"Occluded Video Instance Segmentation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"occubench","name":"OccuBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for MiniMax-M2.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 79.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on gregxmhu.github.io: 42 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oceangym","name":"OceanGym","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GLM-4.5V."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on oceangpt.github.io: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ocnli","name":"OCNLI","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ocr-arena","name":"OCR Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","12 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on ocrarena.ai: 18 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ocr-reasoning","name":"OCR-Reasoning","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (31 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","19 models we track, plus 15 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 65.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 18 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ocr-vqa","name":"OCR-VQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (118 entries)."],["scores.models","pass","80 models we track, plus 38 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for GPT-4.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4.1 (#138)."],["scores.headroom","pass","The best is 82%, under 85%, so it still separates models."],["import.importer","pass","We read the board on OpenCompass: 80 rows for models we track and 38 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ocrbench","name":"OCRBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries), launch posts (34 scores) and system cards (19 scores)."],["scores.models","pass","46 models we track, plus 18 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K2.5 (#76)."],["scores.headroom","info","Saturated. The best is 94%, past 85% since August 2024."],["import.importer","pass","We read the board on huggingface.co: 11 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ocrbench-v2","name":"OCRBench v2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries), launch posts (20 scores) and system cards (14 scores)."],["scores.models","pass","34 models we track, plus 11 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-OCR-1.0."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 74.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on 99franklin.github.io: 15 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 5 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oct-bench","name":"OCT-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries)."],["scores.models","pass","9 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Mini (#74)."],["scores.headroom","pass","The best is 62%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"octocodingbench","name":"OctoCodingbench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (1 score)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for MiniMax-M2.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 36.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"odinw-13","name":"ODinW-13","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (12 scores) and system cards (4 scores)."],["scores.models","pass","13 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.6 Plus (#75)."],["scores.headroom","pass","The best is 51.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"odv-bench","name":"ODV-Bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","7 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Seed2.0 Lite (0428)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 72.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on happyzqy.github.io: 5 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"odysseyarena","name":"OdysseyArena","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","10 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 44.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on yayayacc.github.io: 12 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"odysseyarena-ai-trading","name":"OdysseyArena AI Trading","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","9 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 23.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on yayayacc.github.io: 11 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"odysseyarena-energy-dispatch","name":"OdysseyArena Energy Dispatch","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","9 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 23.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on yayayacc.github.io: 11 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"odysseyarena-repo-system","name":"OdysseyArena Repo System","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","9 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 62.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on yayayacc.github.io: 11 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"odysseyarena-turn-on-lights","name":"OdysseyArena Turn on Lights","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","9 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 28.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on yayayacc.github.io: 11 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"odysseybench","name":"OdysseyBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","7 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 58.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"odysseys-mega-turing","name":"OdysseysMega","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 41%, under 85%, so it still separates models."],["import.importer","pass","We read the board on bench.turing.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"officeprism","name":"OfficePrism","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 76.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 124 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"officeqa","name":"OfficeQA","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (6 scores) and system cards (11 scores)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 86.3%, past 85% since April 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"officeqa-pro","name":"OfficeQA Pro","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries), launch posts (5 scores) and system cards (11 scores)."],["scores.models","pass","23 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 80.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 13 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","2 large gaps of 5 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"officeqa-pro-v2","name":"OfficeQA Pro V2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries) and launch posts (1 score)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 64.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","warn","1 finding disputed or fixed only in a copy: Broken grader, leaked answers, unclear tasks and trips content filters."]]},{"id":"ogs-bench","name":"OGS-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","17 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on opengastronomy.org: 17 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ohsumed","name":"OHSUMED","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oibench","name":"OIBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","11 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for LongCat-Flash-Thinking-2601."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","pass","The best is 47.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ojbench","name":"OJBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries), launch posts (14 scores) and system cards (7 scores)."],["scores.models","pass","23 models we track."],["scores.recent","pass","The newest is from June 2026, for Keye-VL-2.0-30B-A3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 71.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on he-ren.github.io: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ok-vqa","name":"OK-VQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries), launch posts (1 score) and system cards (3 scores)."],["scores.models","pass","3 models we track, plus 18 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Gemma 3 12B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","pass","The best is 60.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on okvqa.allenai.org: 18 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"okapi-arc","name":"Okapi ARC","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"okapi-bench-api-freshness","name":"Okapi Bench: API Freshness","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","34 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 66.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 34 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"okapi-hellaswag","name":"Okapi HellaSwag","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"okapi-mmlu","name":"Okapi MMLU","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries), launch posts (2 scores) and system cards (8 scores)."],["scores.models","pass","9 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Falcon-H1-34B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"okapi-truthfulqa","name":"Okapi TruthfulQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"olam-arena","name":"Olam Labs Multi-Agent Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on olamlabs.ai: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"olmocr-bench","name":"olmOCR-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries) and launch posts (3 scores)."],["scores.models","pass","4 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-OCR-1.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 87.8%, past 85% since October 2026."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"olymmath","name":"OlymMATH","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (28 entries)."],["scores.models","pass","13 models we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-0.6B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#104)."],["scores.headroom","pass","The best is 74.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 13 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"olympiadbench","name":"OlympiadBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for InternVL3.5-241B-A28B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 82.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"olympicarena","name":"OlympicArena","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (29 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","26 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for ERNIE-4.5-300B-A47B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","pass","The best is 75.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on gair-nlp.github.io: 25 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omega-math","name":"OMEGA","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Olmo 3.1 32B Think."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 53.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omicsbench-latchbio-v1","name":"OmicsBench v1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries)."],["scores.models","pass","15 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 40.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on blog.latch.bio: 26 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omni-cloze","name":"Omni-Cloze","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (3 entries), launch posts (3 scores) and system cards (2 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 64.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omni-interactscope","name":"Omni-InteractScope","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","9 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 20.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omni-interactscope-explicit-request","name":"Omni-InteractScope Explicit Request","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","5 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 20.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on yinsongyan.github.io: 5 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omni-interactscope-proactive","name":"Omni-InteractScope Proactive","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","4 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 46.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on yinsongyan.github.io: 4 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omni-math","name":"Omni-MATH","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","13 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Ring-1T."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","pass","The best is 82.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on omni-math.github.io: 9 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omni3d","name":"Omni3D","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 53.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omniact","name":"OmniACT","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on SOTA2: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omniai-ocr-benchmark","name":"OmniAI OCR Benchmark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries) and system cards (1 score)."],["scores.models","pass","5 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for STEP3-VL-10B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 4 Maverick (#153)."],["scores.headroom","pass","The best is 82.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on web.archive.org: 4 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnibench","name":"OmniBench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","17 models we track, plus 18 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for LongCat-Flash-Omni."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#167)."],["scores.headroom","pass","The best is 61.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on m-a-p.ai: 19 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnicad","name":"OmniCAD","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since April 2025."],["import.importer","pass","We read the board on omnieda.com: 40 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnicap-if","name":"OmniCap-IF","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","7 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 80.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on nju-link.github.io: 7 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnidocbench","name":"OmniDocBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores) and system cards (2 scores)."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 6 months ago, in May 2026, for GPT-5.5 Instant. That’s expected for a test replaced by OmniDocBench v1.5."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","warn","The record links a board on github.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnidocbench-1-5","name":"OmniDocBench v1.5","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries), launch posts (26 scores) and system cards (9 scores)."],["scores.models","pass","19 models we track, plus 29 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 12 rows for models we track and 29 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnidocbench-1-6","name":"OmniDocBench v1.6","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (38 entries), launch posts (5 scores) and system cards (1 score)."],["scores.models","pass","11 models we track, plus 28 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 10 rows for models we track and 28 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omniearth-bench","name":"OmniEarth-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","6 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 31.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on omniearth-bench.github.io: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omniechobench","name":"OmniEchoBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","7 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","pass","The best is 43.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on pku-value-lab.github.io: 7 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnigaia","name":"OmniGAIA","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries), launch posts (4 scores) and system cards (2 scores)."],["scores.models","pass","8 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 74%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnigamearena","name":"OmniGameArena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","10 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mxlin043.github.io: 10 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnigui","name":"OmniGUI","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 65.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on omni-gui.github.io: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnilabel","name":"OmniLabel","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on omnilabel.org: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnilayout","name":"OmniLayout","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on omnieda.com: 72 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnilingua-audiomc","name":"Omnilingua-AudioMC","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","4 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","pass","The best is 64%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 17 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnilingua-bench","name":"Omnilingua-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnilingual-asr-corpus","name":"Omnilingual ASR Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnilingual-gaia2","name":"Omnilingual-Gaia2","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Glimmer."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 60.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnimatbench","name":"OmniMatBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","11 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Grok 4.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 11 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnimech","name":"OmniMech","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","8 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on omnieda.com: 38 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnimedvqa","name":"OmniMedVQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","2 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in May 2023, for instructblip-vicuna-7b."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 50.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnimmi","name":"OmniMMI","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","1 model we track, plus 25 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Seed1.8."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 53%, under 85%, so it still separates models."],["import.importer","pass","We read the board on omnimmi.github.io: 25 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omniparsebench","name":"OmniParseBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries) and launch posts (1 score)."],["scores.models","pass","13 models we track, plus 22 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","5 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 12 rows for models we track and 22 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omniphys","name":"OmniPhys","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnirouting","name":"OmniRouting","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 28%, under 85%, so it still separates models."],["import.importer","pass","We read the board on omnieda.com: 91 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnisch","name":"OmniSch","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-122B-A10B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on omnisch.com: 40 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omniscore-bench","name":"OmniScore-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnispatial","name":"OmniSpatial","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries), launch posts (3 scores) and system cards (6 scores)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 58.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnivbench","name":"OmniVBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","5 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on wxliii.github.io: 5 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"omnivideobench","name":"OmniVideoBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","10 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 63.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on omnivideobench.github.io: 8 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"evaluating-user-simulators-arena","name":"On Evaluating User Simulators for Interactive Agents","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oncologybench","name":"OncologyBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries)."],["scores.models","pass","30 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 50.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 50 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oncotriad-qa","name":"OncoTriad-QA","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Gemma 4 E4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 46%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oneig-bench","name":"OneIG-Bench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for GLM-Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kimi-online-experience-moonshot","name":"Online Experience (Moonshot)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 77.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"online-mind2web","name":"Online-Mind2Web","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries), launch posts (10 scores) and system cards (6 scores)."],["scores.models","pass","13 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Fara1.5-27B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Saturated. The best is 92.8%, past 85% since September 2025."],["import.importer","pass","We read the board on huggingface.co: 10 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ood-speech-bengali","name":"OOD-Speech","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (745 entries)."],["scores.models","pass","745 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on kaggle.com: 745 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oolong","name":"OOLONG","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","17 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 31B IT."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on oolongbench.github.io: 17 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oolong-pairs","name":"OOLONG-Pairs","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 92.9%, past 85% since June 2026."],["import.importer","pass","We read the board on primeintellect.ai: 6 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-arabic-llm-leaderboard-v1","name":"Open Arabic LLM Leaderboard v1","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (732 entries)."],["scores.models","pass","57 models we track, plus 675 other entries on its leaderboard."],["scores.recent","info","The newest came 22 months ago, in January 2025, for Qwen2.5-14B-Instruct-1M. That’s expected for a test replaced by Open Arabic LLM Leaderboard v2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 69.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 62 rows for models we track and 675 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-arabic-llm-leaderboard-v2","name":"Open Arabic LLM Leaderboard v2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (151 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","40 models we track, plus 112 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Falcon-Arabic-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 72.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 52 rows for models we track and 112 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-asr-leaderboard","name":"Open ASR Leaderboard","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (25 entries), launch posts (1 score) and system cards (3 scores)."],["scores.models","pass","6 models we track, plus 19 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 19 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-chinese-llm-leaderboard","name":"Open Chinese LLM Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (337 entries)."],["scores.models","pass","35 models we track, plus 302 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for Qwen2-0.5B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2-72B (#180)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 37 rows for models we track and 302 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-cot-leaderboard","name":"Open CoT Leaderboard","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (133 entries)."],["scores.models","pass","32 models we track, plus 101 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for DeepSeek-R1-Distill-Llama-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 17.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 32 rows for models we track and 101 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-ended-questions-hallucination-openai","name":"Open Ended Questions (hallucination)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 2 years ago, in September 2024, for o1-mini. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-flageval-vlm-leaderboard","name":"Open FlagEval VLM Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (51 entries)."],["scores.models","pass","41 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for InternVL3-78B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#104)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 42 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-hebrew-llm-leaderboard","name":"Open Hebrew LLM Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (201 entries)."],["scores.models","pass","88 models we track, plus 113 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 4 31B IT (#101)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 89 rows for models we track and 113 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-japanese-llm-leaderboard","name":"Open Japanese LLM Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (418 entries)."],["scores.models","pass","60 models we track, plus 358 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Reka Flash 3.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is QwQ-32B (#137)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 122 rows for models we track and 358 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-ko-llm-leaderboard","name":"Open Ko-LLM Leaderboard","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1,741 entries)."],["scores.models","pass","18 models we track, plus 1,723 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in June 2024, for Qwen2-7B. That’s expected for a test replaced by Open Ko-LLM Leaderboard 2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Mistral 7B v0.1 (#212)."],["scores.headroom","pass","The best is 53%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 18 rows for models we track and 1,723 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-ko-llm-leaderboard-2","name":"Open Ko-LLM Leaderboard 2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1,166 entries)."],["scores.models","pass","21 models we track, plus 1,145 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Qwen2.5-Coder-14B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is phi-3-medium 14B (#194)."],["scores.headroom","pass","The best is 49.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 24 rows for models we track and 1,145 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-llm-leaderboard-v1","name":"Open LLM Leaderboard v1","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6,797 entries)."],["scores.models","pass","76 models we track, plus 6,721 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in June 2024, for Qwen2-0.5B. That’s expected for a test replaced by Open LLM Leaderboard v2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen2-72B (#180)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 111 rows for models we track and 6,721 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-llm-leaderboard-v2","name":"Open LLM Leaderboard v2","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (4,494 entries)."],["scores.models","pass","101 models we track, plus 4,393 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for QwQ-32B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is QwQ-32B (#137)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 107 rows for models we track and 4,393 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-lmm-reasoning-leaderboard","name":"Open LMM Reasoning Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (82 entries)."],["scores.models","pass","61 models we track, plus 21 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for SAIL-VL-1.6-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 61 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-persian-llm-leaderboard","name":"Open Persian LLM Leaderboard","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","17 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Gemma 3 12B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","pass","The best is 70.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 19 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-portuguese-llm-leaderboard","name":"Open Portuguese LLM Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1,165 entries)."],["scores.models","pass","175 models we track, plus 990 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for Keye-VL-1.5-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 200 rows for models we track and 990 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-prompt-injection","name":"Open Prompt Injection","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-rag-benchmark","name":"Open RAG Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-tabletop-gm-narrative-probe","name":"Open Tabletop GM Narrative Quality Probe","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Gemma 4 31B IT. That’s expected for a test replaced by Open Tabletop GM Narrative Quality Probe v2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemma 4 31B IT (#101)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on reddit.com: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-tabletop-gm-narrative-probe-2","name":"Open Tabletop GM Narrative Quality Probe v2","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries)."],["scores.models","pass","17 models we track, plus 20 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5 397B-A17B (#78)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 17 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-tabletop-gm-session-probe","name":"Open Tabletop GM Session Command Probe","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","9 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 31B IT."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 4 31B IT (#101)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-tts-leaderboard-1-0","name":"Open TTS Leaderboard 1.0.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","3 models we track, plus 24 other entries on its leaderboard."],["scores.recent","info","The newest came 10 months ago, in January 2026, for Qwen3-TTS-12Hz-1.7B-Base. That’s expected for a test replaced by Open TTS Leaderboard 1.1.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on huggingface.co: 3 rows for models we track and 24 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-tts-leaderboard-1-1","name":"Open TTS Leaderboard 1.1.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","3 models we track, plus 31 other entries on its leaderboard."],["scores.recent","info","The newest came 10 months ago, in January 2026, for Qwen3-TTS-12Hz-1.7B-Base. That’s expected for a test replaced by Open TTS Leaderboard 1.2.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on huggingface.co: 3 rows for models we track and 31 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-tts-leaderboard-1-2","name":"Open TTS Leaderboard 1.2.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries)."],["scores.models","pass","3 models we track, plus 34 other entries on its leaderboard."],["scores.recent","info","The newest came 10 months ago, in January 2026, for Qwen3-TTS-12Hz-1.7B-Base. That’s expected for a test replaced by Open TTS Leaderboard 1.3.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on huggingface.co: 3 rows for models we track and 34 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-tts-leaderboard-1-3","name":"Open TTS Leaderboard 1.3.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries)."],["scores.models","pass","3 models we track, plus 42 other entries on its leaderboard."],["scores.recent","info","The newest came 10 months ago, in January 2026, for Qwen3-TTS-12Hz-1.7B-Base. That’s expected for a test replaced by Open TTS Leaderboard 1.4.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on huggingface.co: 3 rows for models we track and 42 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-tts-leaderboard-1-4","name":"Open TTS Leaderboard 1.4.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (47 entries)."],["scores.models","pass","3 models we track, plus 44 other entries on its leaderboard."],["scores.recent","info","The newest came 10 months ago, in January 2026, for Qwen3-TTS-12Hz-1.7B-Base. That’s expected for a test replaced by Open TTS Leaderboard 1.5.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on huggingface.co: 3 rows for models we track and 44 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-tts-leaderboard-1-5","name":"Open TTS Leaderboard 1.5.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (49 entries)."],["scores.models","pass","3 models we track, plus 46 other entries on its leaderboard."],["scores.recent","info","The newest came 10 months ago, in January 2026, for Qwen3-TTS-12Hz-1.7B-Base. That’s expected for a test replaced by Open TTS Leaderboard 1.6.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on huggingface.co: 3 rows for models we track and 46 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"open-tts-leaderboard-1-6","name":"Open TTS Leaderboard 1.6.0","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (66 entries)."],["scores.models","pass","4 models we track, plus 62 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Qwen3-TTS-12Hz-1.7B-Base."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on huggingface.co: 5 rows for models we track and 62 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-agentic-alignment-evals","name":"OpenAI agentic alignment evals","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-cyber-safety-evaluations-incl-daybreak-blue","name":"OpenAI cyber safety evaluations (incl. Daybreak Blue)","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-deployment-simulation","name":"OpenAI Deployment Simulation","summary":{"pass":6,"warn":0,"fail":1,"info":3,"text":"6 of 7 checks pass, 1 problem"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","fail","Unverified. We haven’t checked it against a primary source yet."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (7 scores)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 0.47%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-dynamic-mental-health-benchmarks","name":"OpenAI Dynamic Mental Health Benchmarks","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oeci-openai","name":"OpenAI Epoch Capabilities Index (OECI)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-gpt-red-jailbreak-and-prompt-injection-evals","name":"OpenAI GPT-Red jailbreak and prompt-injection evals","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-internal-ai-self-improvement-evals","name":"OpenAI internal AI self-improvement evals","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-internal-cyber-evaluations","name":"OpenAI internal cyber evaluations","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-monitorability-evals","name":"OpenAI monitorability evals","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-mrcr","name":"OpenAI MRCR","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (6 scores) and system cards (5 scores)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from August 2026, for K-EXAONE 2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 mini (#92)."],["scores.headroom","info","Saturated. The best is 94.4%, past 85% since August 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-mrcr-v2","name":"OpenAI MRCR v2","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (48 entries), launch posts (13 scores) and system cards (3 scores)."],["scores.models","pass","54 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since April 2026."],["import.importer","pass","We read the board on Context Arena: 131 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","2 large gaps of 4 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-performance-in-cases-flagged-by-users","name":"OpenAI performance in cases flagged by users","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-production-benchmarks-and-safe-completion-evals","name":"OpenAI Production Benchmarks and safe-completion evals","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-prs","name":"OpenAI PRs","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (12 scores)."],["scores.models","pass","12 models we track."],["scores.recent","info","The newest came 11 months ago, in December 2025, for GPT-5.2 Codex. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 55%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-research-engineer-interview-coding","name":"OpenAI Research Engineer Interview: Coding","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (7 scores)."],["scores.models","pass","7 models we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for o3. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o3 (#84)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since August 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-research-engineer-interview-mcq","name":"OpenAI Research Engineer Interview: Multiple Choice","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (9 scores)."],["scores.models","pass","9 models we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for ChatGPT agent. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 83%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-user-flagged-hallucination-eval","name":"OpenAI user-flagged hallucination eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Sol (#13)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openai-proof-qa","name":"OpenAI-Proof Q&A","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (9 scores)."],["scores.models","pass","9 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for GPT-5.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 8.00%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openaudiobench","name":"OpenAudioBench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 13 months ago, in October 2025, for LongCat-Flash-Omni. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 80.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openaudiobench-alpacaeval","name":"OpenAudioBench AlpacaEval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Baichuan-Omni-1.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openaudiobench-llama-questions","name":"OpenAudioBench Llama Questions","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Baichuan-Omni-1.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 78.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openaudiobench-reasoning-qa","name":"OpenAudioBench Reasoning QA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Baichuan-Omni-1.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 50%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openaudiobench-triviaqa","name":"OpenAudioBench TriviaQA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Baichuan-Omni-1.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openaudiobench-web-questions","name":"OpenAudioBench Web Questions","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Baichuan-Omni-1.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openbookqa","name":"OpenBookQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries), launch posts (8 scores), system cards (22 scores) and claims Epoch AI collected (41 models)."],["scores.models","pass","57 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Saturated. The best is 96.6%, past 85% since November 2023."],["import.importer","pass","We read the board on github.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openbounds-v2-pilot","name":"OpenBounds V2 Pilot","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Sonnet 5.5 (#9)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"opencanopy","name":"OpenCanopy","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"opencomputer","name":"OpenComputer","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Kimi K2.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 68.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on echo0715.github.io: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"opencpop","name":"Opencpop","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"opendesign-arena","name":"OpenDesign Arena","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","22 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Tencent Hy4."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on open-design.ai: 22 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openearthagent","name":"OpenEarthAgent","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","3 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 44.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openeqa","name":"OpenEQA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for gemini-robotics-er-1.5-preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#174)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on open-eqa.github.io: 2 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openfarm-bioacustics","name":"OpenFARM Bioacoustics","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openfindata","name":"OpenFinData","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openfrontiers-bench","name":"OpenFrontiers-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","pass","The best is 42.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 14 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"opengameeval","name":"OpenGameEval","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","22 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Opus 4.7. That’s expected for a test replaced by OpenGameEval Expanded Eval Set."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 55.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 22 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"opengameeval-debug","name":"OpenGameEval Debug Evals","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 64.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"opengameeval-87","name":"OpenGameEval Expanded Eval Set","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 50.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openhands-index","name":"OpenHands Index","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","33 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 81%, under 85%, so it still separates models."],["import.importer","pass","We read the board on index.openhands.dev: 33 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openhueval","name":"OpenHuEval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"kaggle-opening-lines-arena","name":"Opening Lines Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 17 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openlegalretrieval-de","name":"OpenLegalRetrieval-DE","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","4 models we track, plus 29 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on VAGO Solutions: 4 rows for models we track and 29 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openlex3d","name":"OpenLex3D","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on openlex3d.github.io: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openlmm-spatial-leaderboard","name":"OpenLMM Spatial Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (59 entries)."],["scores.models","pass","51 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 101 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openlmm-subjective-leaderboard","name":"OpenLMM Subjective Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","26 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 26 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openmed-medreason","name":"OpenMed MedReason Evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openrca","name":"OpenRCA","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 34.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openrsi-index","name":"OpenRSI Index","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openrsi-index-ace-playbook-repair","name":"OpenRSI Index ACE Playbook Inspection and Repair","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on index.openrsi.foundation: 2 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openrsi-index-isaaclab-peginsert-reward-search","name":"OpenRSI Index Isaac Lab PegInsert Reward Search","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on index.openrsi.foundation: 2 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openrsi-index-kev-decision-architecture","name":"OpenRSI Index Kev Decision Architecture","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on index.openrsi.foundation: 2 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openrsi-index-long-short-cot-adaptation","name":"OpenRSI Index Learnability-Aware Long/Short CoT Adaptation","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 47.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on index.openrsi.foundation: 2 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openrsi-index-marin-scaling-ladder","name":"OpenRSI Index Marin Scaling Ladder","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on index.openrsi.foundation: 2 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openrsi-index-minference-sparse-prefill","name":"OpenRSI Index MInference 32-Head Sparse Prefill Kernel","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","pass","We read the board on index.openrsi.foundation: 2 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openrsi-index-molmo2-video-pointing","name":"OpenRSI Index Molmo2 Video-Pointing Inference Strategy","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on index.openrsi.foundation: 2 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openrsi-index-molmoweb-context-allocation","name":"OpenRSI Index MolmoWeb Interaction Context Allocation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on index.openrsi.foundation: 2 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openrsi-index-reasonir-curriculum","name":"OpenRSI Index ReasonIR Difficulty Curriculum","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on index.openrsi.foundation: 2 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openrsi-index-sglang-omni-tts-runtime","name":"OpenRSI Index SGLang Omni TTS Runtime","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openscore-string-quartets","name":"OpenScore String Quartets","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openstudio-mcp-1-2-1","name":"OpenStudio-MCP LLM Test Benchmark v1.2.1","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since February 2026."],["import.importer","pass","We read the board on github.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"opensun3d-object-instance-search","name":"OpenSUN3D Open-Vocabulary 3D Object Instance Search","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on opensun3d.github.io: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openthoughts-tblite","name":"OpenThoughts-TBLite","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Kimi K2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K2.5 (#76)."],["scores.headroom","pass","The best is 75.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openthoughts.ai: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openttd-bench","name":"OpenTTD-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openvlm-leaderboard","name":"OpenVLM Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (277 entries)."],["scores.models","pass","174 models we track, plus 103 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 180 rows for models we track and 103 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"openvlm-video-leaderboard","name":"OpenVLM Video Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries)."],["scores.models","pass","29 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for MiMo-VL-7B-RL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 29 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"operator-prompt-injection-scenarios-openai","name":"Operator prompt injection scenarios","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for OpenAI Computer-Using Agent (CUA). Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ophthoquestions","name":"OphthoQuestions ophthalmology board preparation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"opinionqa","name":"OpinionQA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","3 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 4 years ago, in November 2022, for davinci-003."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on proceedings.mlr.press: 3 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"opportunistic-sandbox-escape-anthropic","name":"Opportunistic sandbox escape (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Fable 5.2 (#2)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"opt-bench","name":"OPT-BENCH","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","15 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for DeepSeek-V3.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-235B-A22B-Thinking (#108)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 45 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"opt-rsvg","name":"OPT-RSVG","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"optimbench-meta","name":"OptimBench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","pass","The best is 64.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"optimbench-dispatch","name":"OptimBench Dispatch","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"opus-magnum-bench","name":"Opus Magnum Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on opusmagnumbench.com: 23 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"opus-100","name":"OPUS-100","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"opusparcus","name":"Opusparcus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"or-bench","name":"OR-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries) and system cards (3 scores)."],["scores.models","pass","31 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on huggingface.co: 28 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"or-license","name":"OR-License","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 88.4%, past 85% since August 2026."],["import.importer","pass","We read the board on or-license.com: 11 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"orak","name":"Orak","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","6 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Nemotron 3 Nano 4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on krafton-ai.github.io: 10 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"orangesum","name":"OrangeSum","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (79 entries)."],["scores.models","pass","34 models we track, plus 45 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Nemotron 3 Nano 4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 38.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on EuroEval: 36 rows for models we track and 45 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"orav-bench","name":"ORAV Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"orca-bench-oncall","name":"ORCA-bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 29.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"organic-chemistry-anthropic","name":"Organic Chemistry (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Saturated. The best is 86.2%, past 85% since May 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"organic-chemistry-v2-anthropic","name":"Organic Chemistry V2 (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Fable 5.1."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","pass","The best is 69.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"os-critic-bench","name":"OS-Critic Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","4 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 68.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"os-kairos","name":"OS-Kairos","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","2 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for Qwen2-VL-7B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 29.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"osgym","name":"OSGym","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"osi-bench","name":"OSI-Bench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 30.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ost-bench","name":"OST-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","6 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for GPT-4.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4.1 (#138)."],["scores.headroom","pass","The best is 53.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 12 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"osworkerbench","name":"OSWorkerBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"osworld","name":"OSWorld","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries), launch posts (13 scores), system cards (10 scores) and claims Epoch AI collected (1 model)."],["scores.models","pass","18 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","pass","The best is 78.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"osworld-2-0","name":"OSWorld 2.0","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries), launch posts (18 scores) and system cards (6 scores)."],["scores.models","pass","26 models we track."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 77.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on osworld-v2.xlang.ai: 23 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"osworld-2-0-2026-08-08","name":"OSWorld 2.0 (Aug 2026)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries) and system cards (1 score)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","pass","The best is 77.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on osworld-v2.xlang.ai: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"osworld-2-1","name":"OSWorld 2.1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (7 scores) and system cards (5 scores)."],["scores.models","pass","10 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Muse Ember."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 81.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on osworld-v2.xlang.ai: 9 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"osworld-2-1-offline","name":"OSWorld 2.1 (offline subset)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry), launch posts (4 scores) and system cards (3 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 89.3%, past 85% since November 2026."],["import.importer","pass","We read the board on osworld-v2.xlang.ai: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"osworld-g","name":"OSWorld-G","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries), launch posts (4 scores) and system cards (5 scores)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Fara1.5-27B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 71.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on osworld-grounding.github.io: 3 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"osworld-mcp","name":"OSWorld-MCP","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","9 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 56.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on osworld-mcp.github.io: 18 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"osworld-science","name":"OSWorld-Science","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 73.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on discoailab.github.io: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"osworld-verified","name":"OSWorld-Verified","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (55 entries), launch posts (40 scores) and system cards (19 scores)."],["scores.models","pass","64 models we track, plus 22 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Mage-VL-Cowork."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 86.1%, past 85% since June 2026."],["import.importer","pass","We read the board on os-world.github.io: 74 rows for models we track and 22 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 10 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"otbench","name":"OTBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"otelbench","name":"OTelBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GLM-4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 29%, under 85%, so it still separates models."],["import.importer","pass","We read the board on quesma.com: 14 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"otis-mock-aime-2024-2025","name":"OTIS Mock AIME 2024-2025","summary":{"pass":7,"warn":1,"fail":0,"info":2,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from Epoch AI’s runs (214 models)."],["scores.models","pass","214 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since July 2025."],["import.importer","warn","The record links a board on epoch.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"outboundeval-xbench","name":"OutboundEval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 12 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ovbench","name":"OVBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries), launch posts (4 scores) and system cards (4 scores)."],["scores.models","pass","12 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Flash (#168)."],["scores.headroom","pass","The best is 70%, under 85%, so it still separates models."],["import.importer","pass","We read the board on videochat-online.github.io: 7 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oven","name":"OVEN","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"over-eager-behavior-gui-computer-use-anthropic","name":"Over-eager behavior on GUI computer use tasks (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"overconfidence-command-line-syntax-anthropic","name":"Overconfidence: verifying command-line syntax (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oversearchqa","name":"OverSearchQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oversight-gaming-openai","name":"Oversight gaming","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ambiguous-high-risk-tasks-eval-google","name":"Overstepping boundaries on ambiguous and high-risk tasks","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","pass","The best is 55.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"overtonbench","name":"OvertonBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for DeepSeek-R1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on overtonbench.github.io: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ovo-bench","name":"OVO-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries), launch posts (4 scores) and system cards (4 scores)."],["scores.models","pass","7 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 80.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on joeleelyf.github.io: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ovo-s-bench","name":"OVO-S-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries)."],["scores.models","pass","15 models we track, plus 22 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 59.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on internlm.github.io: 16 rows for models we track and 22 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"owe-bench","name":"OWE-Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 67.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mercor.com: 16 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oxford-flowers-102","name":"Oxford Flowers 102","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oxford-iiit-pet-dataset","name":"Oxford-IIIT Pet Dataset","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"p3b3","name":"P3B3","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries)."],["scores.models","pass","13 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 31B IT."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 39 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"p3d-bench","name":"P3D-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on spatiaos.github.io: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"package-registry-publishing-anthropic","name":"Package registry publishing evaluation (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pact-lech-mazur","name":"PACT","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 models we track."],["scores.recent","pass","The newest is from June 2026, for GLM-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 26 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pact-robot-arm","name":"PACT (robot arm orchestration)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pagentbench-iflytek","name":"PAgentBench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Spark-X2.6-4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 74.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pai-bench","name":"PAI-Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries) and system cards (2 scores)."],["scores.models","pass","7 models we track, plus 13 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 83.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"paintbench","name":"PaintBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","8 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for FLUX.2 [dev]."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 17.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on paintbench.github.io: 8 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"paintskills","name":"PaintSkills","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"paired-symmetry-writer","name":"Paired-symmetry suite","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pairs-parallel-images","name":"PAIRS (Parallel Images)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (43 entries)."],["scores.models","pass","41 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for o3. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","info","Saturated. The best is 97.2%, past 85% since November 2023."],["import.importer","pass","We read the board on VHELM: 46 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"bioweaponization-red-teaming-openai","name":"Pairwise Bioweaponization Campaign","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 60%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pairwise-safety-red-teaming-openai","name":"Pairwise safety red teaming","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 19 months ago, in April 2025, for o3. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pakistan-local-context-llm-suite","name":"Pakistan Local Context LLM Suite","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 0%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 3 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"palmx-2025-general-culture","name":"PalmX 2025 General Culture","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on palmx.dlnlp.ai: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"palmx-2025-islamic-culture","name":"PalmX 2025 Islamic Culture","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on palmx.dlnlp.ai: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"palmyra-x6-enterprise-workflows-writer","name":"Palmyra X6 Enterprise Workflow Evaluations","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"paloma","name":"Paloma","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pam-audio","name":"PAM-Audio","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"panda-cvl","name":"Panda-CVL","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Step 5 Preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 17.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pantograph-frontier-llm-robot-benchmark-pandroid","name":"Pantograph frontier-LLM robot benchmark (Pandroid)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 36.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on pantograph.com: 2 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"paperbench","name":"PaperBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries), launch posts (5 scores) and system cards (11 scores)."],["scores.models","pass","18 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for LongCat-2.5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","info","Saturated. The best is 93%, past 85% since August 2026."],["import.importer","pass","We read the board on raw.githubusercontent.com: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"paperbench-code-dev","name":"PaperBench Code-Dev","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for o1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 43.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"paperbenchx","name":"PaperBenchX","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 14%, under 85%, so it still separates models."],["import.importer","pass","We read the board on unipat.ai: 10 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"paperscope","name":"PaperScope","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","8 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for GPT-5.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.1 (#66)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parade-audio","name":"PARADE Audio","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","11 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#114)."],["scores.headroom","pass","The best is 63.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on AHELM: 14 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parafraseja","name":"Parafraseja","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"paragraph-instruction-following","name":"Paragraph Instruction Following","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parallel-search-capability-leaderboard","name":"Parallel Search Capability Leaderboard","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (58 entries)."],["scores.models","pass","51 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on parallel.ai: 51 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parallelkernelbench","name":"ParallelKernelBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for GLM-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 32.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on together.ai: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parametric-cad-bench","name":"Parametric CAD Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on cadbench.ai: 13 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parametric-cad-bench-v1","name":"Parametric CAD Bench V1","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"paraphrase-stability-benchmark","name":"Paraphrase Stability Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","24 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 24 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"document-ai-benchmark-parashift","name":"Parashift Document AI Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 85.3%, past 85% since September 2026."],["import.importer","pass","We read the board on parashift.ai: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parashoot","name":"ParaShoot","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parcel","name":"PARCEL","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","info","The newest came 11 months ago, in December 2025, for GLM-4.7. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","info","Saturated. The best is 97%, past 85% since July 2025."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pare-bench","name":"Pare-Bench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parlacap-test","name":"ParlaCAP Test Datasets","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","15 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash-Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 15 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parlasent","name":"ParlaSent","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","13 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 13 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parley-labs-negotiation","name":"Parley Labs negotiation tasks","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parliamentbench","name":"ParliamentBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 81%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parsebench","name":"ParseBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (113 entries)."],["scores.models","pass","48 models we track, plus 65 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Tencent Hy4."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 78.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on parsebench.ai: 85 rows for models we track and 65 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parsebench-2","name":"ParseBench-2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (55 entries)."],["scores.models","pass","21 models we track, plus 34 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Muse Ember."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 87%, past 85% since September 2026."],["import.importer","pass","We read the board on parsebench.ai: 22 rows for models we track and 34 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parsinlu","name":"ParsiNLU","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","15 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 15 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parsinlu-mizan","name":"ParsiNLU Mizan Literary Translation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"partbench","name":"PartBench","summary":{"pass":3,"warn":1,"fail":1,"info":5,"text":"3 of 5 checks pass, 1 warning, 1 problem"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","fail","Unverified. We haven’t checked it against a primary source yet."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","pass","The best is 58.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"participation-in-junk-science-anthropic","name":"Participation in junk science (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"partiprompts","name":"PartiPrompts","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on Open Parti Prompts: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"partner-restrictions-compliance-ramp","name":"Partner Restrictions Compliance","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on builders.ramp.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"partnr","name":"PARTNR","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pascal-voc-2007","name":"PASCAL VOC 2007","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pascal-voc-2012","name":"PASCAL VOC 2012","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pascal-context","name":"PASCAL-Context","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"passkey-retrieval","name":"Passkey Retrieval","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for Qwen2.5-14B-Instruct-1M. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Mixtral 8x7B (#199)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since January 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"passport-control","name":"Passport Control","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"past-imo-formal-bytedance","name":"Past IMO (Seed-Prover formal set)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry) and launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Seed-Prover."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 79%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 1 row for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"patchbench-openai","name":"PatchBench (OpenAI)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 79.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"patchcamelyon","name":"PatchCamelyon","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (57 entries)."],["scores.models","pass","1 model we track, plus 56 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 69.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MTEB: 1 row for models we track and 56 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"patcheval","name":"PatchEval","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries) and launch posts (1 score)."],["scores.models","pass","4 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Small."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 83.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"patcheval-verified","name":"PatchEval-Verified","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","3 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 83.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pathmcqa-google","name":"PathMCQA (Google)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"patientagentbench-amazon","name":"PatientAgentBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pattern-labs-cyberoffense-challenges","name":"Pattern Labs cyberoffensive challenges","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"patterneval","name":"PatternEval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (25 entries)."],["scores.models","pass","22 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 44 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pawbench","name":"PAWBench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"paws","name":"PAWS","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"paws-x","name":"PAWS-X","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pazabench","name":"PazaBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pazabench-2","name":"PazaBench v2","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (53 entries)."],["scores.models","pass","4 models we track, plus 49 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Phi-4-multimodal."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on huggingface.co: 4 rows for models we track and 49 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pca-bench","name":"PCA-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","6 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in March 2024, for DeepSeek-VL-7B-Chat."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 12 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pcb-bench-digai","name":"PCB-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pcbbench-protoflow","name":"PCBBench (pcbbench.org)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 89.6%, past 85% since September 2026."],["import.importer","pass","We read the board on pcbbench.org: 8 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pcbworld-bench","name":"PCBWorld-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pclue","name":"pCLUE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries)."],["scores.models","pass","45 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on cluebenchmarks.com: 45 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pct-political-consistency","name":"PCT (Polarized Contrastive Pairs)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (48 entries)."],["scores.models","pass","48 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on dashboard.safe.ai: 48 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pdagent-bench","name":"PDAGENT-BENCH","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","12 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 65.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on fenggev56.github.io: 12 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pdeagent-bench","name":"PDEAgent-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Claude Opus 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 54.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on zeroeclipse00.github.io: 24 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pe-civil-bench","name":"PE Civil Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","8 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for GPT-5.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 87.3%, past 85% since April 2026."],["import.importer","pass","We read the board on komal-blkmmb.github.io: 8 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"audio-event-localization-meta","name":"PE-A-Frame internal audio event localization","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video-to-music-retrieval-meta","name":"PE-AV internal video-to-audio retrieval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fab-pedagogy-benchmark","name":"Pedagogy Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (259 entries)."],["scores.models","pass","216 models we track, plus 43 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 92.9%, past 85% since December 2024."],["import.importer","pass","We read the board on fab-ai.org: 240 rows for models we track and 43 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"oddbit-peer-arena","name":"Peer Arena (oddbit.ai)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","16 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on oddbit.ai: 32 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"peerqa","name":"PeerQA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","10 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for deepseek-r1-0528-qwen3-8b."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 51 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pencil-puzzle-bench","name":"Pencil Puzzle Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (53 entries)."],["scores.models","pass","53 models we track."],["scores.recent","pass","The newest is from June 2026, for GLM-5.2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 97.6%, past 85% since June 2026."],["import.importer","pass","We read the board on ppbench.com: 70 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"penguinscrolls","name":"PenguinScrolls","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and system cards (1 score)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Hunyuan-A13B-Instruct. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","info","Saturated. The best is 87.7%, past 85% since June 2025."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"penn-epilepsy-clinic-notes","name":"Penn epilepsy clinic notes","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"people-making-conversation","name":"People Making Conversation","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Gemini 3.1 Flash Live."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pd-cft","name":"People’s Daily and Children’s Fairy Tale","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"per-doc","name":"Per-DOC","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Llama 2-13B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-70B (#210)."],["scores.headroom","pass","The best is 0.21%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 3 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"per-mpst","name":"Per-MPST","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","3 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Llama 2-13B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-70B (#210)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 3 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"perception-test-mcvqa","name":"Perception Test MCVQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries), launch posts (1 score) and system cards (6 scores)."],["scores.models","pass","7 models we track, plus 41 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 78.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on eval.ai: 1 row for models we track and 41 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"perceptionbench-moonshot","name":"PerceptionBench (Moonshot)","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries), launch posts (6 scores) and system cards (3 scores)."],["scores.models","pass","21 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 80.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 16 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"perceptioncomp","name":"PerceptionComp","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 39.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robocurve-perceptual-attacks","name":"Perceptual Attacks on Embodied AI (Robocurve)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on robocurve.org: 15 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"percor","name":"PerCoR","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries)."],["scores.models","pass","35 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","info","Saturated. The best is 94.3%, past 85% since February 2025."],["import.importer","pass","We read the board on mcinext-mizan-llm-leaderboard.hf.space: 39 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"periapical-lesions-bench","name":"Periapical Lesions Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 18.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on dylantirandaz.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"permembench","name":"PerMemBench","summary":{"pass":6,"warn":2,"fail":0,"info":4,"text":"6 of 8 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5 mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 mini (#92)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"permmlu","name":"PerMMLU","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries)."],["scores.models","pass","35 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 84%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mcinext-mizan-llm-leaderboard.hf.space: 39 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"persian-ifeval","name":"Persian IFEval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries)."],["scores.models","pass","36 models we track, plus 5 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","info","Saturated. The best is 92.6%, past 85% since February 2025."],["import.importer","pass","We read the board on mcinext-mizan-llm-leaderboard.hf.space: 40 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"persian-mt-bench","name":"Persian MT-Bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (41 entries)."],["scores.models","pass","35 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on mcinext-mizan-llm-leaderboard.hf.space: 39 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"persian-nlg","name":"Persian NLG","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (41 entries)."],["scores.models","pass","37 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mcinext-mizan-llm-leaderboard.hf.space: 41 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"persistbench","name":"PersistBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"person-identification-openai","name":"Person Identification evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 19 months ago, in April 2025, for o3. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"personal-agent-bench-micro1","name":"Personal Agent Bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on micro1.ai: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"personal-data-evals-openai","name":"Personal data evaluations","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 21 months ago, in February 2025, for OpenAI deep research. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"personal-superintelligence-index-meta","name":"Personal Superintelligence Index (Meta)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (3 scores) and system cards (3 scores)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for Muse Ember."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-personal-device-assistant-google","name":"Personal-device assistant prompt injection eval","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 98%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"personashot","name":"PersonaShot","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","1 model we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for LTX-2.3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on rain152.github.io: 2 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"personqa-openai","name":"PersonQA","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (9 scores)."],["scores.models","pass","9 models we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for gpt-oss-120b. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o3 (#84)."],["scores.headroom","info","Saturated. The best is 94.3%, past 85% since February 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"perspectivegap","name":"PerspectiveGap","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (35 entries)."],["scores.models","pass","35 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 62%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 35 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pestclef-2026","name":"PestCLEF 2026","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 14 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"petri","name":"Petri","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from July 2026, for Muse Spark 1.1."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"petri-2-0","name":"Petri 2.0","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"petri-bloom","name":"Petri Bloom","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pfbench","name":"PFBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pgg-bench-lech-mazur","name":"PGG-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#104)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 18 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phage-plasmid-coevolution-openai","name":"Phage-plasmid Co-evolution","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (10 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phail","name":"PhAIL","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GR00T N1.6."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 75.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on phail.ai: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phail-droid-everyday","name":"PhAIL DROID: Everyday tasks","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for FLUX 3 Action."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 68.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on phail.ai: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phare","name":"Phare","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (66 entries)."],["scores.models","pass","64 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 83.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on phare.giskard.ai: 65 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pharmabench-micro1","name":"PharmaBench (micro1)","summary":{"pass":9,"warn":2,"fail":0,"info":1,"text":"9 of 11 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","6 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 17.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on micro1.ai: 14 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pheb","name":"PHEB","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phee","name":"PHEE","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phibench","name":"PhiBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (3 scores) and system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Phi-4-Reasoning-plus. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Phi-4 (#160)."],["scores.headroom","pass","The best is 74.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phinc","name":"PHINC","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phmforge","name":"PHMForge","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Opus 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 80.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phoatis","name":"PhoATIS","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on doi.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phomt","name":"PhoMT","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phoneharness-bench","name":"PhoneHarness Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Seed2.0 Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phoneshell-ios-agent-benchmark","name":"PhoneShell iOS Agent Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","info","Saturated. The best is 96.6%, past 85% since September 2025."],["import.importer","pass","We read the board on blolabel.ai: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phybench","name":"PHYBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries), launch posts (3 scores) and system cards (4 scores)."],["scores.models","pass","33 models we track."],["scores.recent","pass","The newest is from July 2026, for Tencent Hy3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 77.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on phybench.cn: 32 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phylogenetics-anthropic","name":"Phylogenetics (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Opus 4.7. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.7 (#31)."],["scores.headroom","pass","The best is 79.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phyre","name":"PHYRE","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phyrubric-bench","name":"PhyRubric-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","3 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Wan2.2-T2V-A14B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"physcapbench","name":"PhysCapBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on hfwang2001.github.io that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"physclosurebench","name":"PhysClosureBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on uncertainty2019.github.io: 7 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"physgym-arena-dr-hard","name":"PhysGym Arena DR-Hard","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ple-google","name":"Physical-Engineering's Last Exam","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","21 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","6 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ple-bench.com: 143 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"physicalrealismbench-u","name":"PhysicalRealismBench-U","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 60.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on reka.ai: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"physicianbench","name":"PhysicianBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries) and system cards (3 scores)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 64.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on healthrex.github.io: 13 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"physics-yale","name":"PHYSICS","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","21 models we track, plus 13 other entries on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for Intern-S1. That’s expected for a test replaced by PHYSICS-Verified."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3-mini (#124)."],["scores.headroom","pass","The best is 59.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 20 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"physics-gre-inflection","name":"Physics GRE (Inflection)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"physics-iq","name":"Physics-IQ","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries) and system cards (1 score)."],["scores.models","pass","4 models we track, plus 13 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Cosmos3-Edge."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 57.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"physics-iq-verified","name":"Physics-IQ Verified","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","12 models we track, plus 13 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Cosmos3-Edge."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on physics-iq-verified.anates.ai: 12 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"physics-yale-verified","name":"PHYSICS-Verified","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","20 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 91.1%, past 85% since July 2026."],["import.importer","pass","We read the board on huggingface.co: 20 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"physicsfinals","name":"PhysicsFinals","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 2 years ago, in May 2024, for Gemini 1.5 Pro. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 1.5 Pro (#174)."],["scores.headroom","pass","The best is 63.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"physiome-ode","name":"Physiome-ODE","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"physvista","name":"PhysVista","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","13 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 85%, past 85% since December 2025."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phyworldbench","name":"PhyWorldBench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"phyx","name":"PhyX","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","13 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 83.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on killthefullmoon.github.io: 10 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pianobench","name":"PianoBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries)."],["scores.models","pass","16 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 93.2%, past 85% since February 2026."],["import.importer","pass","We read the board on arxiv.org: 64 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"picabench","name":"PICABench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","5 models we track, plus 8 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on picabench.github.io: 5 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pico-dsl-v0","name":"Pico DSL v0","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pieverse-agent-prediction-arena","name":"Pieverse Agent Prediction Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on binance.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pigeonholing-partisans-four-word-generation","name":"Pigeonholing Partisans four-word generation","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","6 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for Gemma 2 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-70B (#182)."],["scores.headroom","pass","The best is 51.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 6 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pii-masking","name":"PII Masking","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on app.primeintellect.ai: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pii-masking-300k","name":"PII-Masking-300k","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finmteb-pii-clustering","name":"PiiClustering","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pile-test","name":"Pile-test","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for DeepSeek-V3. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pinchbench","name":"PinchBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (49 entries), launch posts (13 scores) and system cards (6 scores)."],["scores.models","pass","59 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Glimmer."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 Pro (#19)."],["scores.headroom","info","Saturated. The best is 94.9%, past 85% since December 2025."],["import.importer","pass","We read the board on pinchbench.com: 48 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 3 pairs differs by more than 10 points, the largest for GLM-5V-Turbo (+14 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pinchbench-2","name":"PinchBench 2.0","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (60 entries)."],["scores.models","pass","57 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt 74B-A9B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 93.5%, past 85% since April 2026."],["import.importer","pass","We read the board on pinchbench.com: 58 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ping-or-peek-snorkel","name":"Ping or Peek","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 0.59%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pipecat-stt-benchmark","name":"Pipecat Speech-to-Text Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","4 models we track, plus 20 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Muse Voice Transcribe 1.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on pipecat.ai: 4 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"piqa","name":"PIQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries), launch posts (14 scores), system cards (32 scores) and claims Epoch AI collected (58 models)."],["scores.models","pass","82 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GLM-5 (#89)."],["scores.headroom","info","Saturated. The best is 88.7%, past 85% since May 2023."],["import.importer","pass","We read the board on yonatanbisk.com: 1 row for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pirhdy-accompaniment-suggestion","name":"PiRhDy Accompaniment Suggestion","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pirhdy-melody-completion","name":"PiRhDy Melody Completion","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pisa-bench","name":"PISA-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","15 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Qwen3-VL-4B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","pass","The best is 63.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 15 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pitchbench","name":"PitchBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 19.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pitvqa","name":"PitVQA","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pixel2perspective","name":"Pixel2Perspective","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aniruddhsikdar1.github.io: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pixmo-count","name":"PixMo-Count","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (27 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","27 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Keye-VL-2.0-30B-A3B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","info","Saturated. The best is 85.2%, past 85% since September 2024."],["import.importer","pass","We read the board on arxiv.org: 25 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"plaba-abstract-adaptation-2023","name":"PLABA 2023 Abstract Adaptation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"plaba-abstract-adaptation-2024","name":"PLABA 2024 Complete Abstract Adaptation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"plaba-term-replacement-2024","name":"PLABA 2024 Term Replacement","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"places-audio-captions-hindi","name":"Places Audio Captions (Hindi)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"places205","name":"Places205","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"plain-english-summarization-of-contracts","name":"Plain English Summarization of Contracts","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"plaba","name":"Plain Language Adaptation of Biomedical Abstracts","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on nature.com: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"planbench","name":"PlanBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for DeepSeek-R1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-R1 (#130)."],["scores.headroom","pass","The best is 55.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"planbench-xl","name":"PlanBench-XL","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 77.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"planningbench-tencent","name":"PlanningBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","12 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Tencent Hy3 preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 63.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 15 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"plawbench","name":"PLawBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries) and launch posts (1 score)."],["scores.models","pass","17 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 73.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 17 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"playworld","name":"PlayWorld","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"plcworld","name":"PLCWorld","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","5 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 62.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on yunji0516.github.io: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"plm-videobench","name":"PLM-VideoBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","6 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Gemini 2.0 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"plotpoints-round-1","name":"PlotPoints Round 1: Single-Turn Arena","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B. That’s expected for a test replaced by PlotPoints Round 2: Multi-Turn Arena."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Sonnet 4.5 (#81)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on plotlightstudios.com: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"plotpoints-round-2","name":"PlotPoints Round 2: Multi-Turn Arena","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","19 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash. That’s expected for a test replaced by PlotPoints Round 3: Multi-Turn Arena."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.7 (#31)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on plotlightstudios.com: 19 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"plotpoints-round-3","name":"PlotPoints Round 3: Multi-Turn Arena","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries)."],["scores.models","pass","30 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on plotlightstudios.com: 41 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"plotpoints-round-4","name":"PlotPoints Round 4: Willingness and Judgment","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (51 entries)."],["scores.models","pass","39 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Tencent Hy4."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on plotlightstudios.com: 43 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pluginject-openai","name":"PlugInject","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 99.6%, past 85% since December 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pmbench-databricks","name":"PMBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","10 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on databricks.com: 10 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pmc-vqa","name":"PMC-VQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries), launch posts (10 scores) and system cards (1 score)."],["scores.models","pass","10 models we track, plus 11 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 66.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 11 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pmpp-eval","name":"PMPP CUDA Evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pmpp-hard","name":"PMPP-Hard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","12 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pncextract","name":"PNCExtract","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"poem-sentiment","name":"Poem Sentiment","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"poemsum","name":"PoemSum","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"poeta-v2","name":"PoETa v2","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","15 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4.1 (#138)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 15 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"point-bench","name":"Point-Bench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries), launch posts (1 score) and system cards (3 scores)."],["scores.models","pass","19 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Seed1.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 81.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on pointarena.github.io: 17 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pokeagent-challenge","name":"PokeAgent Challenge","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on pokeagent.github.io: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"poker-arena","name":"Poker Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pokebench-rvb","name":"PokéBench: Raw Vision + Buttons","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","18 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on pokebench.tv: 18 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pokebench-tab","name":"PokéBench: Tool Assisted Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (38 entries)."],["scores.models","pass","34 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on pokebench.tv: 37 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pokemon-vgc-bench-ricotoma","name":"Pokémon VGC Bench (Ricotoma)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","14 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on ricotoma.com: 14 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"polaris-healthcare-safety-hippocratic","name":"Polaris Healthcare Safety Benchmarks","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on hippocraticai.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"polaris-bench","name":"Polaris-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 39.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on google-deepmind.github.io: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"polemo-2-0","name":"PolEmo 2.0","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"poleval-2025-gender-inclusive-proofreading","name":"PolEval 2025 Gender-inclusive Proofreading","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on poleval.pl: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"poleval-2025-gender-sensitive-translation","name":"PolEval 2025 Gender-sensitive Translation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on poleval.pl: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"policy-agent-ramp","name":"Policy Agent","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on builders.ramp.com: 33 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"policybench","name":"PolicyBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (52 entries)."],["scores.models","pass","51 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 95.2%, past 85% since April 2026."],["import.importer","pass","We read the board on policybench.org: 51 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"polish-medical-exams","name":"Polish Medical Exams","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","10 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Llama 3.2 1B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 79.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 10 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"poleval-2025-polish-speech-emotion","name":"Polish Speech Emotion Recognition Challenge","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on poleval.pl: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"political-persuasion-parallel-generation-openai","name":"Political Persuasion Parallel Generation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for o3-mini. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 53.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"political-science-llm-benchmark","name":"Political Science LLM Benchmark","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (43 entries)."],["scores.models","pass","42 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 71.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on hannohilbig.com: 42 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"poly-guard-bench","name":"Poly-Guard Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries) and system cards (3 scores)."],["scores.models","pass","3 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"polymath","name":"PolyMath","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (38 entries), launch posts (17 scores) and system cards (11 scores)."],["scores.models","pass","55 models we track."],["scores.recent","pass","The newest is from August 2026, for K-EXAONE 2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.7-Max (#49)."],["scores.headroom","info","Saturated. The best is 86.5%, past 85% since May 2026."],["import.importer","pass","We read the board on qwen-polymath.github.io: 44 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 7 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"popcs","name":"PopCS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pope","name":"POPE","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (214 entries), launch posts (3 scores) and system cards (6 scores)."],["scores.models","pass","153 models we track, plus 65 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for LFM2.5-VL-3B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","info","Saturated. The best is 90.7%, past 85% since May 2023."],["import.importer","pass","We read the board on OpenCompass: 151 rows for models we track and 65 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"popqa","name":"PopQA","summary":{"pass":5,"warn":1,"fail":0,"info":4,"text":"5 of 6 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (7 scores) and system cards (5 scores)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Olmo 3.1 32B Think."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 55.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"portal2bench","name":"Portal2Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kingstonyip.com: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sanskrit-post-ocr-text-correction","name":"Post-OCR Text Correction in Sanskrit","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on aclanthology.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"posterbench","name":"PosterBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on raw.githubusercontent.com: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"posteriorbench","name":"PosteriorBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"posttrainbench","name":"PostTrainBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries), launch posts (5 scores) and system cards (6 scores)."],["scores.models","pass","23 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 74.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on posttrainbench.com: 31 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"posttrainbench-1-1","name":"PostTrainBench 1.1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries) and launch posts (2 scores)."],["scores.models","pass","9 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 41.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on posttrainbench.com: 13 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"posttrainbench-1-2","name":"PostTrainBench 1.2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries) and launch posts (3 scores)."],["scores.models","pass","15 models we track, plus 11 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","5 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 44.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on posttrainbench.com: 24 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"powerbench","name":"PowerBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from November 2026, for Doubao Seed Evolving."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 74.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"powercodebench","name":"PowerCodeBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","12 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Mini (#74)."],["scores.headroom","pass","The best is 68.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 148 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pplx-online-human-evaluation-perplexity","name":"PPLX Online LLM Human Evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ppr10k","name":"PPR10K","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on openaccess.thecvf.com: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ppt-coedit-bench-tencent","name":"PPTCoEditBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ppt-fix-bench-tencent","name":"PPTFixBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pptgen-bench-tencent","name":"PPTGen-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from November 2026, for Tencent Hy4."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 7 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"alyukov-pravda-data-voids","name":"Pravda data voids chatbot audit","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#120)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on misinforeview.hks.harvard.edu: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prbench","name":"PRBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (49 entries) and launch posts (2 scores)."],["scores.models","pass","50 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 61.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 49 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prdbench","name":"PRDBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","4 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GLM-4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 69%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pre-flight","name":"Pre-Flight","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (75 entries)."],["scores.models","pass","74 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 84%, under 85%, so it still separates models."],["import.importer","pass","We read the board on airsidelabs.com: 77 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"precedent-vs-spec","name":"Precedent vs. Spec","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 62.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prediction-arena-arcada","name":"Prediction Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on predictionarena.ai: 18 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"chatgpt-api-prompt-preference-openai","name":"Preference on ChatGPT and API prompts","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 3 years ago, in March 2023, for GPT-4. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-4 (#181)."],["scores.headroom","pass","The best is 70.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"preference-proxy-evaluations","name":"Preference Proxy Evaluations","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","9 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Skywork-Reward-Gemma-2-27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","pass","The best is 68.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 11 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"preference-asr","name":"Preference-ASR","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Phi-4-multimodal."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"premind-bench","name":"PReMind-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 87.3%, past 85% since September 2026."],["import.importer","pass","We read the board on oyt9306.github.io: 8 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prescience","name":"PreScience","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"presentbench","name":"PresentBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","3 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Step 5 Preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 79.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on presentbench.github.io: 9 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pressure-tested-alignment","name":"Pressure-Tested Alignment","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pretraingym","name":"PretrainGym","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"primevul","name":"PrimeVul","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries) and system cards (2 scores)."],["scores.models","pass","3 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Qwen3-Coder-Next. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemma 2 27B (#190)."],["scores.headroom","info","Saturated. The best is 97.1%, past 85% since February 2024."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prinzbench","name":"prinzbench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","26 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 28 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prism-instruction-order-sensitivity","name":"PRISM Instruction Order Sensitivity","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash-Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-30B-A3B-Instruct (#144)."],["scores.headroom","pass","The best is 66%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prism-bench-audio-video","name":"PRISM-Bench (audio and video generation)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prism-bench-image-generation","name":"PRISM-Bench (image generation)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","8 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for BAGEL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on flux-reason-6m.github.io: 18 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"privacylens","name":"PrivacyLens","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries) and system cards (1 score)."],["scores.models","pass","5 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Glimmer."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Sonnet (#187)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"privy-pii","name":"Privy Synthetic PII Protocol Trace Dataset","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries)."],["scores.models","pass","40 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on PII Arena: 40 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prmbench","name":"PRMBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (25 entries)."],["scores.models","pass","3 models we track, plus 22 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Gemini 2.0 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 22 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"proactive-refusals-high-risk-tasks-openai","name":"Proactive refusals of high-risk tasks","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for ChatGPT agent. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 94%, past 85% since January 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"proactive-agent-contact-searches-openai","name":"Proactive Searches","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"proactive-videoqa","name":"ProactiveVideoQA","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for TML-Interaction-Small. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"probsum-2023","name":"ProbSum 2023","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codalab.lisn.upsaclay.fr: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"procbench","name":"ProcBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","8 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Seed2.0 Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Saturated. The best is 96.6%, past 85% since February 2026."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"processbench","name":"ProcessBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (25 entries)."],["scores.models","pass","10 models we track, plus 15 other entries on its leaderboard."],["scores.recent","info","The newest came 23 months ago, in December 2024, for Llama 3.3 70B Instruct. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Saturated. The best is 87.9%, past 85% since September 2024."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prodbench-tencent","name":"ProdBench (Tencent)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Tencent Hy3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 23%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"production-deception-adversarial-openai","name":"Production Deception - Adversarial","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 11 months ago, in December 2025, for GPT-5.2. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"production-jailbreaks-openai","name":"Production Jailbreaks","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for o3-mini. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1 (#118)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"profbench","name":"ProfBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (72 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","74 models we track."],["scores.recent","pass","The newest is from August 2026, for Granite 4.2 30B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 66.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 102 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"programbench","name":"ProgramBench","summary":{"pass":11,"warn":1,"fail":0,"info":0,"text":"11 of 12 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries), launch posts (10 scores) and system cards (12 scores)."],["scores.models","pass","26 models we track."],["scores.recent","pass","The newest is from November 2026, for Qwen4-Max."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 26%, under 85%, so it still separates models."],["import.importer","pass","We read the board on programbench.com: 30 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 6 pairs differs by more than 10 points, the largest for Claude Opus 5 (+18 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"programdistill","name":"ProgramDistill","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 84.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"progress-classification-google","name":"Progress classification (Google DeepMind)","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and launch posts (1 score)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemini Robotics ER 2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 57.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on blog.google: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"progress-bench","name":"Progress-Bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","10 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Qwen3-VL-2B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"project-euler-matharena","name":"Project Euler (MathArena)","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 89%, past 85% since February 2026."],["import.importer","pass","We read the board on matharena.ai: 4 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"projecttest","name":"ProjectTest","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","6 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 82%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 18 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-search-function-calling-openai","name":"Prompt injection attacks in search and function calling","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Saturated. The best is 94.6%, past 85% since June 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-benchmark-environments-openai","name":"Prompt Injection in Benchmark Environments","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 99.8%, past 85% since November 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-model-generated-environments-openai","name":"Prompt Injection in Model-Generated Environments","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 91.7%, past 85% since November 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-live-bug-bounty-anthropic","name":"Prompt injection live bug bounty (Anthropic with Gray Swan)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-robustness-openai","name":"Prompt injection robustness","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (10 scores)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6 Luna."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 99.8%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-adaptive-browser-use-anthropic","name":"Prompt injection robustness against adaptive attackers, browser use (Anthropic)","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries) and system cards (13 scores)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on anthropic.com: 12 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-adaptive-coding-anthropic","name":"Prompt injection robustness against adaptive attackers, coding (Anthropic)","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (14 scores)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-adaptive-computer-use-anthropic","name":"Prompt injection robustness against adaptive attackers, computer use (Anthropic)","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (14 scores)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mt-prompt-injection-test-suite","name":"Prompt injection test suite for LLM-based machine translation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Command A Translate."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-general-tool-use-google","name":"Prompt injection, general tool use","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 91%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-injection-turn-injection-google","name":"Prompt injection, turn injections in tool use","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 86%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-to-product","name":"Prompt to Product","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries)."],["scores.models","pass","38 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 98.2%, past 85% since June 2025."],["import.importer","pass","We read the board on kaggle.com: 38 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-to-bench-ho-16","name":"Prompt-to-Bench HO-16","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prompt-to-bench-16","name":"Prompt-to-Bench-16","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","12 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 27B (#59)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 19 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"proof-news-election-myths","name":"Proof News election misinformation debunking","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in June 2023, for GPT-4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","pass","The best is 0%, under 85%, so it still separates models."],["import.importer","pass","We read the board on proofnews.org: 1 row for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"proofbench","name":"ProofBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (60 entries)."],["scores.models","pass","59 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for DeepSeek V4 Pro 0813."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 78%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 60 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"proofbench-1-1","name":"ProofBench v1.1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (54 entries) and system cards (1 score)."],["scores.models","pass","53 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since June 2026."],["import.importer","pass","We read the board on vals.ai: 53 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"proofnet","name":"ProofNet","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for DeepSeek-Prover-V2-671B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 37.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"proofrank","name":"ProofRank","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on matharena.ai: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"proopf-b","name":"ProOPF-B","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 37.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"propensitybench","name":"PropensityBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (14 entries) and system cards (1 score)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from June 2026, for Muse Spark."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on labs.scale.com: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prophet-arena","name":"Prophet Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries) and launch posts (1 score)."],["scores.models","pass","25 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on prophetarena.co: 43 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prosa","name":"ProSA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","2 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Qwen1.5-14B-Chat."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prost","name":"PROST","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","1 model we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 years ago, in November 2019, for GPT-2 (1.5B)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 30.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 1 row for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prosus-vending-bench-2","name":"Prosus Vending Bench v2","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"protein-design-anthropic","name":"Protein Design (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 42.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"proteingym","name":"ProteinGym","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (97 entries) and system cards (4 scores)."],["scores.models","pass","97 entries on its leaderboard, none a model we track."],["scores.recent","pass","The newest is from September 2026, for Claude Fable 5.1."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on proteingym.org: 97 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"proteinlmbench","name":"ProteinLMBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","11 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for Intern-S1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#172)."],["scores.headroom","pass","The best is 63.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"protocollm","name":"ProtocolLLM","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen3.6 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.6 27B (#88)."],["scores.headroom","pass","The best is 77.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on RTLBench: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"protocolqa-open-ended-openai","name":"ProtocolQA Open-Ended","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (32 scores)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6 Cyber."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 47.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"protocolqa-open-ended-xai","name":"ProtocolQA Open-Ended (xAI)","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","info","Saturated. The best is 87%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"protocols-anthropic","name":"Protocols (Anthropic life sciences)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","pass","The best is 62%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"deepseek-proverbench","name":"ProverBench (DeepSeek)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for DeepSeek-Prover-V2-671B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 59.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 6 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"prowbench","name":"PROWBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","2 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Code World Model (CWM)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on alaya-lab.github.io: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"proxywar","name":"ProxyWar","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 4 (#113)."],["scores.headroom","pass","The best is 39.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"psebench","name":"PSEBench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","12 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for GPT-5.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 95.3%, past 85% since August 2025."],["import.importer","pass","We read the board on labs.scale.com: 12 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pst50","name":"PST50","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on openaccess.thecvf.com: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"psychethicsbench","name":"PsychEthicsBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","4 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Qwen2.5-14B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-8B (#206)."],["scores.headroom","pass","The best is 75.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 4 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"psychobench","name":"PsychoBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"psycomark-conspiracy-detection","name":"PsyCoMark: Conspiracy Detection","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (28 entries)."],["scores.models","pass","28 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codabench.org: 28 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"psycomark-marker-extraction","name":"PsyCoMark: Conspiracy Marker Extraction","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","16 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codabench.org: 16 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"psytar","name":"PsyTAR","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ptcg-bench","name":"PTCG-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on ddhua.tech: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pubhealth","name":"PUBHEALTH","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-public-benefits-bench","name":"Public Benefits Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (50 entries)."],["scores.models","pass","50 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 76.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 50 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"public-tests-hidden-tests","name":"Public Tests, Hidden Tests","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pubmed-causal-language","name":"PubMed causal language use","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pubmed-pico","name":"PubMed-PICO","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pubmedqa","name":"PubMedQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries), launch posts (1 score) and system cards (3 scores)."],["scores.models","pass","6 models we track, plus 21 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Reka Core."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 82%, under 85%, so it still separates models."],["import.importer","pass","We read the board on pubmedqa.github.io: 3 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pucrs-inpatient-fall-detection","name":"PUCRS inpatient fall detection","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pulsebench-tab","name":"PulseBench-Tab","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","4 models we track, plus 14 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 4 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pumac-2024","name":"PUMaC 2024","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pumavos","name":"PUMaVOS","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on SOTA2 Research: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"puneval","name":"PunEval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Claude 3 Opus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 36 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pushworld","name":"PushWorld","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"putnam-2024","name":"Putnam 2024","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for DeepSeekMath-V2. That’s expected for a test replaced by Putnam 2025."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 98.3%, past 85% since November 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"putnam-2025","name":"Putnam 2025","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Nemotron 3 Ultra."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MathArena: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"putnam-axiom","name":"Putnam-AXIOM","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries)."],["scores.models","pass","2 models we track, plus 16 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for o1-preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-preview (#149)."],["scores.headroom","pass","The best is 41.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"putnam-like-google","name":"Putnam-like","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o4-mini (#95)."],["scores.headroom","info","Saturated. The best is 94.5%, past 85% since March 2025."],["import.importer","pass","We read the board on github.com: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"putnambench","name":"PutnamBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries), launch posts (5 scores) and system cards (5 scores)."],["scores.models","pass","16 models we track, plus 26 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for DiffusionGemma 26B A4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V4-Flash (#87)."],["scores.headroom","info","Saturated. The best is 87.9%, past 85% since December 2025."],["import.importer","pass","We read the board on trishullab.github.io: 13 rows for models we track and 26 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 4 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"puzzleworld","name":"PuzzleWorld","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","6 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 18%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pvd-bench","name":"PVD Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pwn-ctf-anthropic","name":"Pwn CTF challenges (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 71.4%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pwnbench-v0-1","name":"PWNBench v0.1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 63.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Fireworks AI: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pws-decisions","name":"PWS-Decisions","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pws-finance","name":"PWS-Finance","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on parallel.ai: 5 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pws-health","name":"PWS-Health","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on parallel.ai: 6 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pws-legal","name":"PWS-Legal","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on parallel.ai: 7 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"py-bug-trace-level-3","name":"Py Bug Trace Level 3","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"pyats-bench","name":"pyATS-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"q-bench","name":"Q-Bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","10 models we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for Qwen-VL-Plus."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 73.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 10 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"q-bench-image-pair-perception","name":"Q-Bench Image-Pair Perception","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"q-bench-single-image-perception","name":"Q-Bench Single-Image Perception","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"q-bench-video","name":"Q-Bench-Video","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","3 models we track, plus 14 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for GPT-4o mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o mini (#178)."],["scores.headroom","pass","The best is 52.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"q2d-chunk","name":"Q2D-Chunk","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"q2d-web","name":"Q2D-Web","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","3 models we track, plus 16 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Qwen3-Embedding-0.6B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 79.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 3 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qa4mre","name":"QA4MRE","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qag-360k","name":"QAG-360K","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qalb-2014","name":"QALB 2014","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qalb-2015","name":"QALB 2015","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qampari","name":"QAMPARI","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qasc","name":"QASC","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qasper","name":"Qasper","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries), launch posts (1 score) and system cards (6 scores)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Phi-4 Mini."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 41.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qbqtc","name":"QBQTC","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qcircuitbench","name":"QCircuitBench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Llama 3-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-8B (#206)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qevd-fit-coach","name":"QEVD-FIT-COACH","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on varworkshop.github.io: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qf-bench","name":"QF-Bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 61.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on qfbench.com: 9 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qias-2025-islamic-inheritance-reasoning","name":"QIAS 2025 Islamic Inheritance Reasoning","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on sites.google.com: 15 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qias-2025-islamic-knowledge-assessment","name":"QIAS 2025 Islamic Knowledge Assessment","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on sites.google.com: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qimma","name":"QIMMA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (94 entries)."],["scores.models","pass","44 models we track, plus 50 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 27B (#59)."],["scores.headroom","pass","The best is 69.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 50 rows for models we track and 50 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qivd","name":"QIVD","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (19 entries), launch posts (4 scores) and system cards (2 scores)."],["scores.models","pass","8 models we track, plus 15 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 69.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on q-ivd.github.io: 8 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qmsum","name":"QMSum","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qodo-code-review-benchmark","name":"Qodo AI Code Review Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on qodo.ai: 7 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"quac","name":"QuAC","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries), launch posts (1 score) and system cards (12 scores)."],["scores.models","pass","1 model we track, plus 30 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-70B (#182)."],["scores.headroom","pass","The best is 51.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on quac.ai: 30 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"quadruped-rl-task-anthropic","name":"Quadruped reinforcement learning task (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (8 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qualcomm-interactive-cooking","name":"Qualcomm Interactive Cooking","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#112)."],["scores.headroom","pass","The best is 23.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on varworkshop.github.io: 3 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"quality","name":"QuALITY","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries), launch posts (1 score) and system cards (7 scores)."],["scores.models","pass","11 models we track, plus 13 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in November 2024, for Llama 3.1-405B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Saturated. The best is 90.5%, past 85% since March 2024."],["import.importer","pass","We read the board on nyu-mll.github.io: 4 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"quant-arena","name":"Quant-Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on lab.einsia.ai: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"quantbench-openai","name":"QuantBench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for o1. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 60%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"quanteval","name":"QuantEval","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","9 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 86%, past 85% since September 2025."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"quantsightbench","name":"QuantSightBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on quantsightbench.com: 11 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"quartets-audio-to-score","name":"Quartets Audio-to-Score Transcription","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on futuremirex.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"quartets-mirex-2026","name":"QuartetsMIREX 2026 Audio-to-Score Transcription","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on futuremirex.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"questflow-finance-benchmark","name":"Questflow Finance Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since August 2026."],["import.importer","pass","We read the board on next.questflow.ai: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"questflow-traders-arena","name":"Questflow Traders Arena","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on next.questflow.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"quixbugs","name":"QuixBugs","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"quran-qa-2023-passage-retrieval","name":"Qur’an QA 2023 Passage Retrieval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"quran-qa-2023-reading-comprehension","name":"Qur’an QA 2023 Reading Comprehension","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qvhighlights","name":"QVHighlights","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries), launch posts (3 scores) and system cards (5 scores)."],["scores.models","pass","5 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Nova 2.0 Pro Preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 77.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwen-chinese-human-evaluation","name":"Qwen Chinese human evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 3 years ago, in September 2023, for Qwen-14B. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Qwen-14B (#214)."],["scores.headroom","pass","The best is 30.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwen-in-house-chinese-eval","name":"Qwen in-house Chinese automatic evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for Qwen-Turbo. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Qwen2-72B (#180)."],["scores.headroom","pass","The best is 69.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwen-in-house-english-eval","name":"Qwen in-house English automatic evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for Qwen-Turbo. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Qwen2-72B (#180)."],["scores.headroom","pass","The best is 72.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voice-chat-in-house-qwen","name":"Qwen in-house voice-chat benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 20 months ago, in March 2025, for Qwen2.5-Omni-7B. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cockpit-routing-benchmark-qwen","name":"Qwen-Audio-Agent In-House Cockpit Benchmark","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smart-cockpit-benchmark-qwen","name":"Qwen-Audio-Agent Smart Cockpit Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen3.8 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","info","Saturated. The best is 98.8%, past 85% since June 2026."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwen-image-bench","name":"Qwen-Image-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","13 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Flash Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 13 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"cover-song-generation-qwen","name":"Qwen-Music Cover Song Generation","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"text-to-music-generation-qwen","name":"Qwen-Music Text-to-Music Generation","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwen2-multilingual-safety-eval","name":"Qwen2 multilingual safety evaluation","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in July 2024, for Qwen2-72B. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"forced-alignment-qwen","name":"Qwen3 forced alignment benchmarks","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"asr-internal-transcription-qwen","name":"Qwen3-ASR internal transcription benchmarks","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwenblenderbench","name":"QwenBlenderBench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (1 score)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 69.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on qwen.ai: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwenchinesebench","name":"QwenChineseBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Qwen 3.6 Max (Preview). Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Qwen 3.6 Max (Preview) (#73)."],["scores.headroom","pass","The best is 84%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwenclawbench","name":"QwenClawBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries) and launch posts (6 scores)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 64.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on skylenage-ai.github.io: 12 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwenqoderbench","name":"QwenQoderBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and launch posts (1 score)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 63.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on qwen.ai: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwenreactbench","name":"QwenReactBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and launch posts (1 score)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on qwen.ai: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwensvg","name":"QwenSVG","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (7 entries) and launch posts (2 scores)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on qwen.ai: 7 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwensvgbench","name":"QwenSVGBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and launch posts (1 score)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on qwen.ai: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwenswebench","name":"QwenSWEBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and launch posts (2 scores)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 27B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 86.3%, past 85% since June 2026."],["import.importer","pass","We read the board on qwen.ai: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwenvision2code","name":"QwenVision2Code","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and launch posts (1 score)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on qwen.ai: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwenvisualoffice","name":"QwenVisualOffice","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (1 score)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 44.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on qwen.ai: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwenwebbench","name":"QwenWebBench","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Qwen3.6 27B. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Qwen 3.6 Max (Preview) (#73)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwenwebdev","name":"QwenWebDev","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (2 scores)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on qwen.ai: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"qwenworldbench","name":"QwenWorldBench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (2 scores)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 62.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on qwen.ai: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"r-bench-robustness","name":"R-Bench (real-world robustness)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and system cards (2 scores)."],["scores.models","pass","2 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Nemotron Nano 12B v2 VL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 75.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 6 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"constraint-bench-labelbox","name":"R-ConstraintBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 79.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labelbox.com: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"race","name":"RACE","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries) and system cards (8 scores)."],["scores.models","pass","8 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Saturated. The best is 94.2%, past 85% since August 2025."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"race-h","name":"RACE-H","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (4 scores) and system cards (5 scores)."],["scores.models","pass","7 models we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for DeepSeek-V3. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","info","Saturated. The best is 92.9%, past 85% since July 2023."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"race-m","name":"RACE-M","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (1 score)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 23 months ago, in December 2024, for DeepSeek-V3. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","pass","The best is 77%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"racebias-race-based-medicine","name":"RaceBias (Race-Based Medicine)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"radiobench","name":"RadioBench","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 97%, past 85% since April 2026."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"radiological-nuclear-expert-knowledge-openai","name":"Radiological and Nuclear Expert Knowledge","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 21 months ago, in February 2025, for GPT-4.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 74%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"radle","name":"Radiology's Last Exam (RadLE)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5. That’s expected for a test replaced by RadLE 2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 30%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"radle-2-0","name":"RadLE 2.0","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","12 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on crashlab.in: 12 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"radnlp-2024-staging","name":"RadNLP 2024 Lung Cancer Staging","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 12 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"radnlp-2024-segmentation","name":"RadNLP 2024 Radiology Report Segmentation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"radsum23","name":"RadSum23","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"raft-real-world-annotated-few-shot-tasks","name":"RAFT (Real-world Annotated Few-shot Tasks)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 5 years ago, in March 2021, for GPT-Neo-2.7B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ragbench-galileo","name":"RAGBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ragtruth","name":"RAGTruth","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 77.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"railhead","name":"Railhead","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ramen-bench","name":"Ramen Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","24 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on bitbin.de: 92 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ramp-accounting-bench","name":"Ramp Accounting Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 61.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.ramp.com: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stack-accounting-benchmark-ramp","name":"Ramp Stack Accounting Benchmark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 62.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on builders.ramp.com: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ramp-swe-bench","name":"Ramp SWE-Bench v1","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 models we track."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 88.5%, past 85% since June 2026."],["import.importer","pass","We read the board on labs.ramp.com: 26 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ramp-swe-bench-2","name":"Ramp SWE-Bench v2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","22 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 76.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.ramp.com: 38 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"random-complex-tasks-longcat","name":"Random Complex Tasks (LongCat)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 10 months ago, in January 2026, for LongCat-Flash-Thinking-2601. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 35.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"randombench","name":"RandomBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries)."],["scores.models","pass","37 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 37 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"range-repair-agent","name":"Range Repair Agent","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rapidata-multi-step-image-edit","name":"Rapidata Multi-Step Image Editing (Identity Drift)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","6 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-Image-2.6 Sunburst."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on benchmark.ai: 6 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rarediseasebench-latchbio","name":"RareDiseaseBench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 43.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 31 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rauh-bundestag-sentiment","name":"Rauh Bundestag sentence sentiment validation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ravdess","name":"RAVDESS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ravnursson-faroese-speech","name":"Ravnursson Faroese Speech and Transcripts","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on aclanthology.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"razavi-bench","name":"Razavi-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (73 entries)."],["scores.models","pass","63 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 98.3%, past 85% since April 2026."],["import.importer","pass","We read the board on razavi-bench.tokenzhang.com: 63 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rbench","name":"RBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries) and system cards (1 score)."],["scores.models","pass","12 models we track, plus 18 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Cosmos3-Edge."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 58.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 12 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rbench-m","name":"RBench-M","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","13 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Qwen3-VL-30B-A3B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 69.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on evalmodels.github.io: 13 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rcm-bench-1","name":"RCM-Bench-1","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 94.2%, past 85% since February 2026."],["import.importer","pass","We read the board on eng.commure.com: 27 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rcts-to-scale-nudge-forecasts","name":"RCTs to Scale nudge forecasts","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rd-tablebench","name":"RD-TableBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries) and system cards (1 score)."],["scores.models","pass","1 model we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Nemotron Nano 12B v2 VL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 57.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on reducto.ai: 8 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"re-bench","name":"RE-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"re-bench-subset-anthropic","name":"RE-Bench subset (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rebaltica-latvian-russian-chatbot-sources","name":"Re:Baltica Latvian and Russian chatbot sources","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on en.rebaltica.lv: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"react-native-evals","name":"React Native Evals","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","27 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 89%, past 85% since June 2026."],["import.importer","pass","We read the board on rn-evals.vercel.app: 27 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reactbench","name":"ReactBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","29 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 54.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on reactbench.com: 87 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reacthuman","name":"ReactHuman","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 63.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reactionbench","name":"ReactionBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.7 Flash (#26)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on yibozh.github.io: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reactsim-bench","name":"ReactSim-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"read-my-aura","name":"Read My Aura","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fab-read-aloud-assessment","name":"Read-aloud Assessment Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","7 models we track, plus 15 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on fab-ai.org: 7 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"readability-perplexity","name":"Readability (Perplexity)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 21 months ago, in February 2025, for sonar. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 85.9%, past 85% since February 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"real-bench","name":"REAL","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","13 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Claude Opus 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 41.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on realevals.ai: 14 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"real-pocqi","name":"Real Point-of-Care Queries","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is -1.50%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rw-voice-eq","name":"Real World VoiceEQ","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","5 models we track, plus 34 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 5 rows for models we track and 34 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"real-prompts-human-eval-zai","name":"Real-scenario prompts human evaluation (Z.ai)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GLM-4.5. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"real-swe","name":"Real-SWE","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 38.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on web.archive.org: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"real-swe-1-1","name":"Real-SWE 1.1","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 56.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on withspecific.com: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realtime-av-interaction-meituan","name":"Real-time Audio-Visual Interaction Evaluation","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 13 months ago, in October 2025, for LongCat-Flash-Omni. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"real-time-reasoning-gym","name":"Real-Time Reasoning Gym","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3.2 (#91)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"real-world-audio-recording-openai","name":"Real-World Audio Recording","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for GPT-Live-Transcribe."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"real-world-finance-anthropic","name":"Real-World Finance (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 70%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"real-world-finance-v2-anthropic","name":"Real-World Finance v2 (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"real-world-vulnerability-discovery-google","name":"Real-world vulnerability discovery","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 86%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"real2sim","name":"Real2Sim","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on real2sim.bot: 8 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"real5-omnidocbench","name":"Real5-OmniDocBench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-OCR-1.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 93.7%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realbench-verilog","name":"RealBench (Verilog)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","3 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for o1-preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-preview (#149)."],["scores.headroom","pass","The best is 6.65%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realcommbench","name":"RealCommBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on wwwwwli.github.io: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realcompanion","name":"RealCompanion","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 70.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realestate10k","name":"RealEstate10K","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realhall","name":"RealHall","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realharm-giskard","name":"RealHarm","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realitybench","name":"realitybench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realitytest-aisi","name":"RealityTest","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realkie-fcc-verified","name":"RealKIE-FCC Verified","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores) and system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Nova 2.0 Pro Preview."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 67%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realm","name":"REALM","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for π0-FAST."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 53.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realm-financial-micro1","name":"Realm Financial Reasoning","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","pass","The best is 56.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on micro1.ai: 17 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realmpathology","name":"RealmPathology","summary":{"pass":9,"warn":2,"fail":0,"info":1,"text":"9 of 11 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 78%, under 85%, so it still separates models."],["import.importer","pass","We read the board on micro1.ai: 13 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realtime-bench","name":"Realtime-Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 42.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on realtimeagents.org: 21 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realtoxicityprompts","name":"RealToxicityPrompts","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (6 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Apertus 70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realworkdecision-micro1","name":"RealWorkDecision","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 43.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on micro1.ai: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"realworldqa","name":"RealWorldQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries), launch posts (28 scores) and system cards (19 scores)."],["scores.models","pass","46 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","info","Saturated. The best is 88.5%, past 85% since February 2026."],["import.importer","pass","We read the board on AGI-Eval: 8 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reasoning-behavior-classifiers-anthropic","name":"Reasoning behavior classifiers (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Claude Opus 4. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reasoning-dial","name":"Reasoning Dial","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Haiku 5.5 (#20)."],["scores.headroom","info","Saturated. The best is 99.6%, past 85% since October 2026."],["import.importer","pass","We read the board on kaggle.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reasoning-extraction-red-teaming-openai","name":"Reasoning extraction red-teaming","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reasoning-gym","name":"Reasoning Gym","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reasoning-bias-reporting-quality","name":"Reasoning-Bias-Reporting-Quality","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reasoning-judge-bench","name":"ReasoningJudgeBench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reasonseg","name":"ReasonSeg","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reasonvos","name":"ReasonVOS","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","15 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 15 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reasonzoo","name":"ReasonZoo","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for DeepSeek-R1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-R1 (#122)."],["scores.headroom","pass","The best is 61%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"recipe1m-image-and-recipe-retrieval","name":"Recipe1M image and recipe retrieval","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on pic2recipe.csail.mit.edu: 4 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"recode-h","name":"RECODE-H","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 84.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 30 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"recognition-of-model-written-user-turns-anthropic","name":"Recognition of model-written user turns (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Mythos Preview. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"recon-sixtyfour","name":"RECON","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","14 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 36.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sixtyfour.ai: 14 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"record","name":"ReCoRD","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 17 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in May 2023, for PaLM 2-L. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is PaLM 2-L (#198)."],["scores.headroom","info","Saturated. The best is 93.8%, past 85% since May 2023."],["import.importer","pass","We read the board on sheng-z.github.io: 17 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"recovery-bench-letta","name":"Recovery-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"recreationbench","name":"RecreationBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries) and launch posts (3 scores)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 58.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on recreation-bench.cc: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"red-team-attack-simulator","name":"Red Team Attack Simulator","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"red-team-defense-simulator","name":"Red Team Defense Simulator","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 20 months ago, in March 2025, for GPT-4o. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o mini (#178)."],["scores.headroom","info","Saturated. The best is 85%, past 85% since March 2025."],["import.importer","pass","We read the board on github.com: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rtn-pairwise-safety-comparison-openai","name":"Red Teaming Network pairwise safety comparison","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 21 months ago, in February 2025, for OpenAI deep research. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 59.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"red-pen-bench","name":"Red-Pen Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Haiku 5.5 (#20)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"red-teaming-resistance-haize","name":"Red-Teaming Resistance Benchmark","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"redcode","name":"RedCode","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reddit-impacts","name":"Reddit-Impacts","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 20 months ago, in March 2025, for GPT-4o. That’s expected for a test replaced by RedditImpacts 2.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codalab.lisn.upsaclay.fr: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reddit-impacts-2","name":"RedditImpacts 2.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codabench.org: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"redlinebench","name":"RedlineBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 50.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on micro1.ai: 4 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"redlinebench-v2-fa1zvn","name":"RedlineBench v2 (Fa1zvn)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"redstonebench","name":"RedstoneBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on redstone-bench.adamholter.chatgpt.site: 47 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"redtape","name":"Redtape","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"redteam-arena","name":"RedTeam Arena","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ref-davis17","name":"Ref-DAVIS17","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on FindTrack: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ref-l4","name":"Ref-L4","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (24 entries), launch posts (2 scores) and system cards (3 scores)."],["scores.models","pass","6 models we track, plus 21 other entries on its leaderboard."],["scores.recent","info","The newest came 10 months ago, in January 2026, for GLM-4.6V. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 89.5%, past 85% since July 2025."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ref-youtube-vos","name":"Ref-YouTube-VOS","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on competitions.codalab.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ref-zom","name":"Ref-ZOM","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on openaccess.thecvf.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"refadv-s","name":"RefAdv-S","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries) and launch posts (1 score)."],["scores.models","pass","21 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 80.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ref-adv.github.io: 30 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"refav","name":"RefAV","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eval.ai: 14 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"refchecker","name":"RefChecker","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"refcoco-refcoco-plus-refcocog","name":"RefCOCO / RefCOCO+ / RefCOCOg","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (24 scores) and system cards (15 scores)."],["scores.models","pass","29 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek-V4.1-Flash Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.5 397B-A17B (#78)."],["scores.headroom","info","Saturated. The best is 95%, past 85% since March 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"refinebench","name":"RefineBench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"refspatial-bench","name":"RefSpatial-Bench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (12 scores) and system cards (6 scores)."],["scores.models","pass","16 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Cosmos3-Edge. That’s expected for a test replaced by RefSpatial-Expand-Bench."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.5 397B-A17B (#78)."],["scores.headroom","pass","The best is 73.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"refspatial-expand-bench","name":"RefSpatial-Expand-Bench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"refspatial-expand-bench-location","name":"RefSpatial-Expand-Bench Location","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 2 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"refspatial-expand-bench-placement","name":"RefSpatial-Expand-Bench Placement","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 2 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"refunq","name":"RefuNQ","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","4 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for Mistral-7B-Instruct-v0.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","pass","The best is 53.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"refusal-to-assist-with-ai-safety-rnd-anthropic","name":"Refusal to assist with AI safety R&D (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"refusalbench-nous","name":"RefusalBench (Nous Research)","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","18 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Hermes 4.3 36B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 74.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 21 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"system-jailbreaks-xai","name":"Refusals + System Jailbreak (xAI)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Grok 4.20. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.20 (#61)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"user-jailbreaks-xai","name":"Refusals + User Jailbreak (xAI)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Grok 4.20. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.20 (#61)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"regexeval","name":"RegexEval","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on s2e-lab.github.io: 4 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"regurgitation-evals-openai","name":"Regurgitation Evaluations","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 23 months ago, in December 2024, for o1. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"relate-audio-text","name":"RELATE","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"relbench","name":"RelBench","summary":{"pass":4,"warn":2,"fail":0,"info":4,"text":"4 of 6 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from July 2026, for Grok 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4.5 (#47)."],["scores.headroom","pass","The best is 38.4%, under 85%, so it still separates models."],["import.importer","warn","The record links a board on star-project.stanford.edu that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"relytoolbench","name":"RelyToolBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","5 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o mini (#178)."],["scores.headroom","pass","The best is 77.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 16 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"remi","name":"ReMI","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries), launch posts (2 scores) and system cards (4 scores)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for STEP3-VL-10B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","pass","The best is 67.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"remote-labor-index","name":"Remote Labor Index","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","19 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","5 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 41.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 20 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"remuq","name":"ReMuQ","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"repcount-a","name":"RepCount-A","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for TML-Interaction-Small. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"replibench-aisi","name":"RepliBench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"replicationbench","name":"ReplicationBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 22%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"repobench","name":"RepoBench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for Codestral. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 38%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"repochat-arena","name":"RepoChat Arena","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"repoinject-v1","name":"RepoInject v1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reponormbench","name":"RepoNormBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V4-Flash (#87)."],["scores.headroom","pass","The best is 68.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"repoqa","name":"RepoQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries) and system cards (4 scores)."],["scores.models","pass","26 models we track, plus 10 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Step 3.5 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Saturated. The best is 92.6%, past 85% since February 2024."],["import.importer","pass","We read the board on evalplus.github.io: 22 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reportbench","name":"ReportBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Claude Sonnet 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reprobait-securebio","name":"ReproBAIT","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 85%, past 85% since June 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"research-decision-benchmark","name":"Research Decision Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","pass","The best is 36.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"research-plan-generation-meta","name":"Research Plan Generation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"research-sandbagging-anthropic","name":"Research sandbagging (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"research-eval-reka","name":"Research-Eval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries) and launch posts (2 scores)."],["scores.models","pass","5 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Grok 4.1 Fast."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4 (#86)."],["scores.headroom","pass","The best is 66%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"researchclawbench","name":"ResearchClawBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","23 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on internscience.github.io: 37 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"researchcodebench","name":"ResearchCodeBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","22 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 64.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 23 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"researchrubrics","name":"ResearchRubrics","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries), launch posts (6 scores) and system cards (3 scores)."],["scores.models","pass","7 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for LongCat-2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 81%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"researchtastebench-meta","name":"ResearchTasteBench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","pass","The best is 64.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"resolve","name":"Resolve","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Gemma 4 31B IT. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 Mini (#74)."],["scores.headroom","info","Saturated. The best is 87.5%, past 85% since October 2025."],["import.importer","pass","We read the board on kaggle.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"respacebench","name":"ReSpaceBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (33 entries)."],["scores.models","pass","17 models we track, plus 16 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 50.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 18 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"respecting-auto-review-openai","name":"Respecting Auto-Review","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (12 scores)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"respecting-warnings-openai","name":"Respecting Warnings","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (12 scores)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"respin-s1","name":"RESPIN-S1.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on papers.nips.cc: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"retailbench","name":"RetailBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Grok 4.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on linghuazhang01.github.io: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"retrace","name":"ReTrace","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 63.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 18 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"retrobench-v0-4","name":"Retrobench v0.4","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 99.9%, past 85% since September 2026."],["import.importer","pass","We read the board on retrobench.ai: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"retrybudget-24-policy-and-json-contracts","name":"RetryBudget-24: Policy and JSON Contracts","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#120)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rev-ctf-anthropic","name":"Rev CTF challenges (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since November 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"revbench-polyvor","name":"RevBench (Polyvor Labs)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 50%, under 85%, so it still separates models."],["import.importer","pass","We read the board on hub.harborframework.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reversal-bench","name":"REVERSAL-BENCH","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reverse-wikipedia-paragraphs","name":"Reverse Wikipedia Paragraphs","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reviewbench-github","name":"ReviewBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","4 models we track, plus 13 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 73.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on review-bench.ai: 8 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"revisited-oxford-and-paris","name":"Revisited Oxford and Paris","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (31 entries)."],["scores.models","pass","31 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on openaccess.thecvf.com: 31 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reward-hack-bench-islo","name":"Reward Hack Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reward-hack-prone-coding-tasks-anthropic","name":"Reward-hack-prone coding tasks (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (8 scores)."],["scores.models","pass","8 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"reward-seeking-contrastive-belief-updates","name":"Reward-seeking via contrastive belief updates","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rewardbench","name":"RewardBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (180 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","50 models we track, plus 132 other entries on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for Hermes 4 405B. That’s expected for a test replaced by RewardBench 2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V3 (#155)."],["scores.headroom","info","Saturated. The best is 95.1%, past 85% since May 2024."],["import.importer","pass","We read the board on huggingface.co: 48 rows for models we track and 132 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rewardbench-2","name":"RewardBench 2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (71 entries)."],["scores.models","pass","42 models we track, plus 29 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Skywork-Reward-V2-Llama-3.1-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 84.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 42 rows for models we track and 29 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rewind-attacker-anthropic","name":"Rewind-attacker cyber adversarial robustness (Anthropic)","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rexbench","name":"RExBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Claude Opus 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","pass","The best is 50%, under 85%, so it still separates models."],["import.importer","pass","We read the board on rexbench.com: 70 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rgb","name":"RGB","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in February 2024, for Qwen1.5-72B. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rgraph-eval","name":"RGraph-Eval","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Cosmos3-Nano."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rh-swe-bench","name":"RH SWE-bench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rh-swe-bench-v1","name":"RH SWE-bench v1","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Qwen3.6 27B. That’s expected for a test replaced by RH SWE-bench."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 63.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ricechem","name":"RiceChem","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","6 models we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Gemma 1.1 7B IT."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Phi-2 (#225)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rich","name":"RICH","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"riemann-bench","name":"Riemann-bench","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries), launch posts (4 scores) and system cards (4 scores)."],["scores.models","pass","24 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","7 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 84%, under 85%, so it still separates models."],["import.importer","pass","We read the board on surgehq.ai: 21 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"right-answer-wrong-reason-ml-judgment-calls","name":"Right Answer, Wrong Reason: ML Judgment Calls","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"riichi-orbit-mahjong-bot-open-practice","name":"RIICHI ORBIT mahjong bot comparison (open practice)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on note.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"riichi-orbit-mahjong-bot-two-stage","name":"RIICHI ORBIT mahjong bot comparison (two-stage practice)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on note.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"riosworld","name":"RiOSWorld","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","7 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on yjyddq.github.io: 7 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rise","name":"RISE","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for MiniMax-M2.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 64.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"risebench","name":"RISEBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries) and system cards (1 score)."],["scores.models","pass","14 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Qwen-Image-Edit-2509. That’s expected for a test replaced by RISEBench++."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 11.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 13 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"risebench-plus-plus","name":"RISEBench++","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","22 models we track, plus 17 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Flash Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 50.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 22 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"riskrubric","name":"RiskRubric","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (38 entries)."],["scores.models","pass","35 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on riskrubric.ai: 40 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rivabench","name":"RivaBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rl-rollout-alignment-faking-audit-google","name":"RL rollout audit for beyond-episode drives","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rlbench","name":"RLBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on CodeSOTA: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rmbench","name":"RMBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","1 model we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for π0.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 10.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rms-aqa","name":"RMS-AQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","3 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 36.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on rmsaqachallenge.github.io: 3 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rmu-echr","name":"RMU:ECHR","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rng-bench","name":"RNG-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 49.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"roadmapbench","name":"RoadmapBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries) and launch posts (1 score)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from September 2026, for Step 5 Preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 54.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on unipat.ai: 12 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"roark-live-bench-1-3","name":"Roark Live Bench v1.3","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (929 entries)."],["scores.models","pass","929 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on benchmarks.roark.ai: 929 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-robo-bench","name":"Robo-bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 32.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 11 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"roboarcade","name":"RoboARCade","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 51.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on roboarcade.org: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"roboarena","name":"RoboArena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","3 models we track, plus 21 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for π0.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on robo-arena.github.io: 3 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robocasa","name":"RoboCasa","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for GR00T N1.7."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 70.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robocasa-gr1-tabletop","name":"RoboCasa GR1 Tabletop Tasks","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries), launch posts (4 scores) and system cards (1 score)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GR00T N1.7."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 50%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robocasa-mem","name":"RoboCasa-Mem","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for π0.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 16%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robocasa365","name":"RoboCasa365","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","4 models we track, plus 20 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GR00T N1.6."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 23.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on robocasa.ai: 4 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robochallenge","name":"RoboChallenge","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","3 models we track, plus 19 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for π0.5. That’s expected for a test replaced by RoboChallenge Table30 v2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 42.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on robochallenge.ai: 3 rows for models we track and 19 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robochallenge-table30-v2","name":"RoboChallenge Table30 v2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (65 entries)."],["scores.models","pass","1 model we track, plus 64 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for π0.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 0.73%, under 85%, so it still separates models."],["import.importer","pass","We read the board on robochallenge.ai: 1 row for models we track and 64 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robocoliseum","name":"RoboColiseum","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","9 models we track, plus 22 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 76.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on robocoliseum.ai: 9 rows for models we track and 22 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robocolosseum","name":"RoboColosseum","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on robocolosseum.ai: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robocurve-physical-tool-use-benchmark-unidentified","name":"Robocurve physical tool-use benchmark (unidentified)","summary":{"pass":1,"warn":2,"fail":1,"info":6,"text":"1 of 4 checks pass, 2 warnings, 1 problem"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","fail","Unverified. We haven’t checked it against a primary source yet."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robocurve-simple-tasks-llm-control-of-yam-arms","name":"Robocurve simple tasks (LLM control of YAM arms)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 52.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on openai.robocurve.org: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robodojo","name":"RoboDojo","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (72 entries) and launch posts (1 score)."],["scores.models","pass","7 models we track, plus 66 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 64.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on robodojo-benchmark.com: 6 rows for models we track and 66 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robodojo-rc-tier-1","name":"RoboDojo-RC Tier 1","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","12 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 65.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on robocurve.org: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robodx","name":"RoboDx","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since April 2026."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"roboeval","name":"RoboEval","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","2 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for MolmoAct 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 44.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on robo-eval.github.io: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"roboflow-100-vl","name":"Roboflow 100-VL","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","1 model we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Qwen2.5-VL-72B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 7.60%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robogate","name":"RoboGate","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","2 models we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GR00T N1.6."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 0%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"roboharm","name":"Roboharm","summary":{"pass":7,"warn":1,"fail":1,"info":3,"text":"7 of 9 checks pass, 1 warning, 1 problem"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","fail","Unverified. We haven’t checked it against a primary source yet."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 20%, under 85%, so it still separates models."],["import.importer","pass","We read the board on robocurve.org: 3 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robolab","name":"RoboLab","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","8 models we track, plus 15 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Praxis-1."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 46.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on research.nvidia.com: 14 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robomemarena","name":"RoboMemArena","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for π0.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 21.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on robomemarena.github.io: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"roborewardbench","name":"RoboRewardBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","22 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on crfm.stanford.edu: 23 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robospatial-home","name":"RoboSpatial-Home","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries), launch posts (1 score) and system cards (3 scores)."],["scores.models","pass","6 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Cosmos3-Edge."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 73.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robocurve-rci","name":"Robot Capabilities Index (Robocurve)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"roboteam","name":"RoboTeam","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 80.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robotic-origami-challenge-iros-2026","name":"Robotic Origami Challenge (IROS 2026)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robots-evolved-gui-lessons","name":"Robots Evolved GUI lessons","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robots-txt-benchmark","name":"Robots.txt Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robotwin-1-0","name":"RoboTwin 1.0","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robotwin-2-0","name":"RoboTwin 2.0","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries)."],["scores.models","pass","2 models we track, plus 34 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for π0.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 58.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on robotwin-platform.github.io: 2 rows for models we track and 34 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robotworld","name":"RobotWorld","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 19.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on robotworldai.github.io: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"roboworld","name":"RoboWorld","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on gym-anything.github.io: 3 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robust-if-bench","name":"Robust IF Bench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for MAI-Code-1-Flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 61.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robust-multi-view-depth","name":"Robust Multi-view Depth","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"robust-kbench","name":"robust-kbench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rocs-mt","name":"RoCS-MT","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rocs-mt-2","name":"RoCS-MT v2","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (56 entries)."],["scores.models","pass","10 models we track, plus 46 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for GPT-4.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4.1 (#138)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 10 rows for models we track and 46 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"roi-calculator","name":"ROI Calculator (dsm)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rok-fortress","name":"ROK-FORTRESS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from August 2026, for K-EXAONE 2.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 89.5%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"role-play-bench-minimax","name":"Role-play Benchmark (MiniMax)","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for MiniMax-M2-her."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rolebench","name":"RoleBench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rolemrc","name":"RoleMRC","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 79.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rollercoastertycoonbench","name":"RollerCoasterTycoonBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on vals.ai: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"roman-urdu-code-switch-benchmark","name":"Roman Urdu Code-Switch Benchmark","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on kaggle.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rome-flageval","name":"ROME","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","20 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on flageval-baai.github.io: 29 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"room-across-room","name":"Room-Across-Room (RxR)","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"roomelsa-shrec-2025","name":"ROOMELSA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aichallenge.hcmus.edu.vn: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rostrans","name":"ROSTrans","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 10.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rota-bench","name":"RoTA-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","8 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 17%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 10 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rotom-champ-bench","name":"Rotom Champ Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (43 entries)."],["scores.models","pass","30 models we track, plus 13 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on rotomarena.org: 31 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rotom-gen9-bench","name":"Rotom Gen9 Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","8 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Haiku 5.5 (#20)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on rotomarena.org: 8 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"roughcut-bench","name":"RoughCut-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on whalesong-zrs.github.io: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"routerbench","name":"RouterBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rp2k","name":"RP2K","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rpc-bench","name":"RPC-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","14 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GLM-4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 20 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rpeval-el-boudouri","name":"RPEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 62.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rqa-70k","name":"RQA-70K weather radar forecast evaluation","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 52.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rrg24","name":"RRG24","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rrn","name":"RRN","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rscc-disaster-events","name":"RSCC","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","3 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for InternVL3-8B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on bili-sakura.github.io: 9 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rsi-arena","name":"RSI Arena","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rsi-bench-scale","name":"RSI Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on rsi-benchmark.com: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rsi-control-arena","name":"RSI Control Arena","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rsi-index-openai","name":"RSI Index (OpenAI)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rsi-exam","name":"RSI-Exam","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","20 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Qwen4-Max."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 53.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on rsi-exam.ai: 20 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rsi-inference-bench","name":"RSI-Inference Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on rsi-inference.com: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rsibench-data","name":"RSIBench-Data","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 32.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on rsibench.co: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rsicd","name":"RSICD","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rsieval","name":"RSIEval","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rslivebench-uav","name":"RSLiveBench-UAV","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (30 entries)."],["scores.models","pass","10 models we track, plus 20 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on rslivebench.space: 10 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rsna-knee-auc10","name":"RSNA Knee AUC10","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","27 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 27 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rsna-knee-llm-label-extraction","name":"RSNA Knee LLM Label Extraction","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rtaste","name":"rTaste","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rte","name":"RTE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (263 entries) and launch posts (1 score)."],["scores.models","pass","3 models we track, plus 261 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in July 2023, for Claude 2. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 2 (#192)."],["scores.headroom","info","Saturated. The best is 92.8%, past 85% since October 2019."],["import.importer","pass","We read the board on gluebenchmark.com: 2 rows for models we track and 261 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rteb","name":"RTEB","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","2 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Qwen3-Embedding-4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 69.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 2 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rtl-opt","name":"RTL-OPT","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen3.6 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.6 27B (#88)."],["scores.headroom","pass","The best is 72.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rtl-repo","name":"RTL-Repo","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","8 models we track, plus 17 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Kimi K2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","pass","The best is 38.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on TuRTLe: 8 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rtllm","name":"RTLLM","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rtllm-2","name":"RTLLM 2.0","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen3.6 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.6 27B (#88)."],["scores.headroom","pass","The best is 60%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rtlscout-bench-2","name":"RTLScout Bench (suite-v2)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3.1 Pro (#17)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on plex1.github.io: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rtp-lx","name":"RTP-LX","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rtsgamebench","name":"RTSGameBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rtvlm","name":"RTVLM","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries) and system cards (2 scores)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Phi-4-multimodal."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ruc-tt07-32k","name":"RuC-tt07-32k","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for DeepSeek-V3.1-Terminus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is gpt-oss-120b (#126)."],["scores.headroom","pass","The best is 76.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 12 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rule2drc","name":"Rule2DRC","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 67.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on jinukkim.me: 24 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ruler","name":"RULER","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (44 entries), launch posts (16 scores) and system cards (35 scores)."],["scores.models","pass","51 models we track, plus 27 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Nemotron 3 Ultra (#85)."],["scores.headroom","info","Saturated. The best is 99%, past 85% since November 2023."],["import.importer","pass","We read the board on github.com: 17 rows for models we track and 27 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 9 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ruler-v2","name":"RULERv2","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Qwen3-235B-A22B-Instruct. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3-235B-A22B-Thinking (#108)."],["scores.headroom","info","Saturated. The best is 89.2%, past 85% since April 2025."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rules-llm","name":"RuLES","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (134 entries)."],["scores.models","pass","30 models we track, plus 104 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in February 2024, for Claude 3 Opus. That’s expected for a test replaced by RuLES 2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 9.54%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 38 rows for models we track and 104 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rules-llm-2","name":"RuLES 2.0","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (134 entries)."],["scores.models","pass","30 models we track, plus 104 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Claude 3 Opus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 9.54%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 38 rows for models we track and 104 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ruletrek","name":"RuleTrek","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","pass","The best is 55.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rumedbench","name":"RuMedBench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rundkast","name":"Rundkast","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"runebench","name":"RuneBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (80 entries)."],["scores.models","pass","79 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on maxbittker.github.io: 110 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"runway-model-router-benchmark","name":"Runway Model Router Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rvg-bench","name":"RVG-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","9 models we track, plus 13 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 49.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on fyrisk.github.io: 9 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rvms-bench","name":"RVMS-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rwe-bench","name":"RWE-bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","11 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 62%, under 85%, so it still separates models."],["import.importer","pass","We read the board on xplore.ai: 15 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rwsearch","name":"RWSearch","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from July 2026, for LongCat-2.0."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 82%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"s-ember","name":"S-EMBER","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 54.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"s-eval","name":"S-Eval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries)."],["scores.models","pass","19 models we track, plus 17 other entries on its leaderboard."],["scores.recent","info","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V3.2 (#91)."],["scores.headroom","info","Saturated. The best is 88.8%, past 85% since August 2025."],["import.importer","pass","We read the board on s.alibaba.com: 21 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"s3gym","name":"S3Gym","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sa-1b-hard-human-pose-evaluation-meta","name":"SA-1B Hard human pose evaluation","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sa-37-interactive-segmentation","name":"SA-37 interactive segmentation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"saarland-midi-audio-piano","name":"Saarland MIDI-Audio Piano Music","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"saas-bench","name":"SaaS-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 31.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on unipat.ai: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sac-bench","name":"SAC-Bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Qwen3-Omni-30B-A3B. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 91.3%, past 85% since September 2025."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"high-severity-implicit-intent-openai","name":"Safe response to high-severity prompts","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-safe-teen","name":"SAFE-Teen","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from October 2026, for Mistral Large 4 Preview."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on vals.ai: 36 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-safe-teen-health-advice","name":"SAFE-Teen Health advice","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from October 2026, for Mistral Large 4 Preview."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on vals.ai: 18 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-safe-teen-relationship-boundaries","name":"SAFE-Teen Relationship boundaries","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from October 2026, for Mistral Large 4 Preview."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on vals.ai: 18 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-safe-teen-urgent-safety","name":"SAFE-Teen Urgent safety","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from October 2026, for Mistral Large 4 Preview."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on vals.ai: 18 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"safelibero","name":"SafeLIBERO","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for π0.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 57.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"safepyramid","name":"SafePyramid","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 54.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on bytedance.github.io: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"safety-inhouse-minimax","name":"Safety (MiniMax in-house)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for MiniMax-Text-01. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 90.9%, past 85% since January 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"safety-modes-cohere","name":"Safety mode controllability (Cohere)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Cohere Command A. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"safety-scene-vision-benchmark","name":"Safety-Scene Vision Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"safetybench","name":"SafetyBench","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GLM-4.5-Air. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 89.9%, past 85% since August 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-sage","name":"SAGE (Vals)","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (81 entries)."],["scores.models","pass","81 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 56.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 91 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-sage-1-1","name":"SAGE v1.1 (Vals)","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","30 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 91.3%, past 85% since February 2026."],["import.importer","pass","We read the board on vals.ai: 29 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sahm","name":"SAHM","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","12 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 80.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mbzuai-nlp.github.io: 12 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"japanese-english-translation-sakana","name":"Sakana Japanese-English translation evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sakana-kana-letter-reading-order-set","name":"Sakana kana-letter reading-order set","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"salad-bench","name":"SALAD-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries) and launch posts (3 scores)."],["scores.models","pass","16 models we track, plus 21 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Granite 4.1 30B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-3.5 Turbo (#200)."],["scores.headroom","info","Saturated. The best is 97.3%, past 85% since September 2023."],["import.importer","pass","We read the board on adwardlee.github.io: 13 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"salesbench-insurance","name":"SalesBench (Insurance Sales)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"salesbench-100","name":"SalesBench-100","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sam-3d-artist-objects","name":"SAM 3D Artist Objects","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sam-3d-body-synthetic-pose-evaluation-meta","name":"SAM 3D Body synthetic pose evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sam-3d-objects-human-preference-evaluation-meta","name":"SAM 3D Objects human preference evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sam-audio-judge-evaluation-meta","name":"SAM Audio Judge evaluation","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sam-audio-bench","name":"SAM Audio-Bench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"samromur-21-05","name":"Samrómur 21.05","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"samromur-children","name":"Samrómur Children","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"samskritam-bench-v0","name":"Samskritam Bench v0","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 99.8%, past 85% since September 2026."],["import.importer","pass","We read the board on samskritam.ai: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"samskritam-bench-v1","name":"Samskritam Bench v1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 89.4%, past 85% since September 2026."],["import.importer","pass","We read the board on samskritam.ai: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sandbox-bench-openai","name":"Sandbox Bench","summary":{"pass":7,"warn":0,"fail":0,"info":3,"text":"All 7 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 45.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sandbox-escape-evaluation-anthropic","name":"Sandbox escape evaluation (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sandbox-escape-bench-aisi","name":"SandboxEscapeBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sandhikosh","name":"SandhiKosh","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sarena","name":"SArena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","19 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Intern-S1-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 4 (#113)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on hmwang2002.github.io: 18 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sas-bench","name":"SAS-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","8 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for MiMo-7B-RL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 67.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sat-spatial-aptitude","name":"SAT (Spatial Aptitude Training)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries) and system cards (1 score)."],["scores.models","pass","3 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for gemini-robotics-er-1.5-preview."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 76.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arijitray.com: 2 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sat-ebrw","name":"SAT Evidence-Based Reading & Writing","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 3 years ago, in March 2023, for GPT-4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sat-math","name":"SAT Math","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sat-reading","name":"SAT Reading","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","pass","The best is 74.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sata-bench","name":"SATA-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (32 entries)."],["scores.models","pass","21 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for gpt-oss-120b."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 75.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 21 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"satbench","name":"SATBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries)."],["scores.models","pass","15 models we track, plus 5 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Qwen3-1.7B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o4-mini (#95)."],["scores.headroom","info","Saturated. The best is 89.3%, past 85% since April 2025."],["import.importer","pass","We read the board on aclanthology.org: 15 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"satlidar-canopy-height-meta","name":"SatLidar canopy height evaluation","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"satnet-dsn-scheduling","name":"SatNet","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"saudade-pt","name":"SAUDADE-PT","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"save-bench-meta","name":"SAVE-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Saturated. The best is 90.7%, past 85% since July 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"savu-bench","name":"SAVU-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","7 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.6 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.6 Flash (#45)."],["scores.headroom","pass","The best is 55.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sayfullina-soft-skill-spans","name":"Sayfullina Soft Skill Span Extraction","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on ceur-ws.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sb10k","name":"SB10k","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sc-comics","name":"SC-CoMIcs","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scadbench","name":"SCADBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on scadbench.com: 19 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scala","name":"ScaLA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (216 entries)."],["scores.models","pass","4 models we track, plus 212 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 57.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MTEB: 4 rows for models we track and 212 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scanbench-baichuan","name":"ScanBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Baichuan-M3-235B. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 73.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scannet","name":"ScanNet","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scannet-plus-plus-v2","name":"ScanNet++ v2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaldir.vc.in.tum.de: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scannet-1500","name":"ScanNet-1500","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on SOTA2: 16 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scarfbench","name":"ScarfBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on kaggle.com: 11 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scbench-latchbio-singlecellbench","name":"scBench (LatchBio SingleCellBench)","summary":{"pass":11,"warn":1,"fail":0,"info":0,"text":"11 of 12 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (27 entries) and system cards (7 scores)."],["scores.models","pass","27 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 63.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 41 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 6 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scbench-latchbio-singlecellbench-original","name":"scBench (original study)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. That’s expected for a test replaced by scBench (LatchBio SingleCellBench)."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 52.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scbench-long","name":"scBench-Long","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 36.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 45 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scbench-long-original","name":"scBench-Long (original study)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","10 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. That’s expected for a test replaced by scBench-Long."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 25.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 15 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scenefun3d","name":"SceneFun3D","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on scenefun3d.github.io: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scholarcatalyst","name":"ScholarCatalyst","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","5 models we track, plus 11 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Fable 5.1."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","pass","The best is 53%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scholarqabench","name":"ScholarQABench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","2 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Llama 3-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-70B (#182)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sci-pipeline-v1","name":"Sci Pipeline v1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sci-agent-exam","name":"SCI-AGENT-EXAM","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on skylenage-ai.github.io: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sciarena","name":"SciArena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (38 entries)."],["scores.models","pass","34 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Olmo 3.1 32B Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on sciarena.allen.ai: 34 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scibench","name":"SciBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Llama 3-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-8B (#206)."],["scores.headroom","pass","The best is 8.45%, under 85%, so it still separates models."],["import.importer","pass","We read the board on scibench-ucla.github.io: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scicap","name":"SciCap","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scicode","name":"SciCode","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries), launch posts (41 scores) and system cards (26 scores)."],["scores.models","pass","67 models we track."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 59.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on scicode-bench.github.io: 20 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scidocs","name":"SciDocs","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"science-gym-chem","name":"Science Sim Chemistry","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sciences-last-exam","name":"Science's Last Exam","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on sciences-last-exam.org: 9 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scienceagentbench","name":"ScienceAgentBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for o1-preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-preview (#149)."],["scores.headroom","pass","The best is 42.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 34 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sciencebenchmark-sql","name":"ScienceBenchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in August 2023, for T5-Large."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 23%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sciencebenchmark.cloudlab.zhaw.ch: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scienceboard","name":"ScienceBoard","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","17 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 68%, under 85%, so it still separates models."],["import.importer","pass","We read the board on qiushisun.github.io: 28 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scienceqa","name":"ScienceQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (78 entries), launch posts (1 score), system cards (2 scores) and claims Epoch AI collected (17 models)."],["scores.models","pass","22 models we track, plus 74 other entries on its leaderboard."],["scores.recent","info","The newest came 20 months ago, in March 2025, for Phi-4-multimodal. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Saturated. The best is 97.5%, past 85% since May 2024."],["import.importer","pass","We read the board on scienceqa.github.io: 4 rows for models we track and 74 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sciencevision","name":"ScienceVision","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from November 2026, for Doubao Seed Evolving."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on unipat.ai: 12 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scienceworld","name":"ScienceWorld","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sfe-scientists-first-exam","name":"Scientists' First Exam (SFE)","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (4 scores) and system cards (4 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for Intern-S2-397B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 60.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scifact","name":"SciFact","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sciknoweval","name":"SciKnowEval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","9 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Claude Sonnet 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scimmir","name":"SciMMIR","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","5 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for mPLUG-Owl2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 6 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scipredict","name":"SciPredict","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","12 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 25.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 12 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sciq","name":"SciQ","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scireasoner-bench","name":"SciReasoner evaluation suite","summary":{"pass":4,"warn":0,"fail":1,"info":5,"text":"4 of 5 checks pass, 1 problem"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","fail","Unverified. We haven’t checked it against a primary source yet."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Intern-S2-397B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 63.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scitaste-bench","name":"SciTaste-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 28 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scits","name":"SciTS","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Intern-S1-Pro."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sciuniverse","name":"SciUniverse","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 45.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on c5r.net: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scivid","name":"SciVid","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 12 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scone-bench-anthropic","name":"SCONE-bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","7 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Claude Opus 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on anthropic.com: 7 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scopecheck-evidence-before-numbers","name":"ScopeCheck: evidence before numbers","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Nano."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Nano (#90)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"score-consistency-and-robustness","name":"SCORE (Consistency and Robustness)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scrabblebench","name":"ScrabbleBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","14 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on sunnybala.github.io: 14 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"screen2words","name":"Screen2Words","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"screenasr-bench","name":"ScreenASR-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","5 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Doubao Seed 2.0 Lite."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"screenspot","name":"ScreenSpot","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries), launch posts (8 scores) and system cards (8 scores)."],["scores.models","pass","12 models we track, plus 5 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Nemotron 3 Nano Omni 30B-A3B. That’s expected for a test replaced by ScreenSpot-v2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 95.4%, past 85% since January 2025."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"screenspot-pro","name":"ScreenSpot-Pro","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (95 entries), launch posts (28 scores) and system cards (15 scores)."],["scores.models","pass","40 models we track, plus 84 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 92.7%, past 85% since December 2025."],["import.importer","pass","We read the board on gui-agent.github.io: 12 rows for models we track and 84 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 4 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"screenspot-v2","name":"ScreenSpot-v2","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries), launch posts (9 scores) and system cards (12 scores)."],["scores.models","pass","14 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Glimmer."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 95.2%, past 85% since April 2025."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"screensuite","name":"ScreenSuite","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Qwen2.5-VL-32B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 56%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scribbleedit","name":"ScribbleEdit","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","1 model we track, plus 8 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scrolls","name":"SCROLLS","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sct-bench","name":"SCT-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for o1-preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-preview (#149)."],["scores.headroom","pass","The best is 70.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"scuba","name":"SCUBA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 53.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sfrcua.github.io: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sd-qa","name":"SD-QA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sdbench-microsoft","name":"SDBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sdr-bench","name":"SDR-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","6 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 55.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on behavior-in-the-wild.github.io: 12 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sea-helm","name":"SEA-HELM","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (95 entries)."],["scores.models","pass","75 models we track, plus 20 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","5 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on leaderboard.sea-lion.ai: 75 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sea-helm-audio","name":"SEA-HELM Audio","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seal-adversarial-robustness","name":"SEAL Adversarial Robustness","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on labs.scale.com: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seal-coding","name":"SEAL Coding","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","26 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for o3-mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on labs.scale.com: 26 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seal-multilingual","name":"SEAL Multilingual","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","19 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for o3-mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on labs.scale.com: 19 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seal-precise-instruction-following","name":"SEAL Precise Instruction Following","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","18 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for DeepSeek-R1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on labs.scale.com: 18 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seal-showdown","name":"SEAL Showdown","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5 Chat."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on scale.com: 13 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seal-0","name":"Seal-0","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries), launch posts (7 scores) and system cards (2 scores)."],["scores.models","pass","19 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-122B-A10B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K2.5 (#76)."],["scores.headroom","pass","The best is 57.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 22 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"search-triggering-eval-google","name":"Search triggering","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 88%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-sec-claim-verification-400","name":"SEC filing claim verification (400 items)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 99.5%, past 85% since October 2025."],["import.importer","pass","We read the board on vals.ai: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sec-bench","name":"SEC-bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 18%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sec-bench.github.io: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sec-bench-pro","name":"SEC-bench Pro","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries), launch posts (7 scores) and system cards (15 scores)."],["scores.models","pass","17 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 90.5%, past 85% since September 2026."],["import.importer","pass","We read the board on sec-bench.github.io: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sec-bench-pro-260617","name":"SEC-bench Pro 260617","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 58.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sec-bench.github.io: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seccodebench","name":"SecCodeBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (1 score)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.7-Max (#49)."],["scores.headroom","pass","The best is 71.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on alibaba.github.io: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"secit-bench","name":"SecIT Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from October 2026, for Mistral Large 4."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 82.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on cribl.io: 34 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"secqa","name":"SecQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","2 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in December 2023, for Mistral-7B-Instruct-v0.2. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 90%, past 85% since October 2023."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"secrespond","name":"SecRespond","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (23 entries)."],["scores.models","pass","23 models we track."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Pro (#19)."],["scores.headroom","pass","The best is 79%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 23 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"secretpatch","name":"SecretPatch","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Gemma 2 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 2 27B (#190)."],["scores.headroom","pass","The best is 72%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"secure-and-defensive-coding-anthropic","name":"Secure and defensive coding (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Fable 5.1."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"securebio-vmqa-single-choice","name":"SecureBio VMQA, single choice","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 61.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"securecodereview-xai","name":"SecureCodeReview (xAI)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","pass","The best is 58.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"secureswe-openai","name":"SecureSWE (OpenAI)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 65.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"security-configuration-verification","name":"Security Configuration Verification","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sedgewick-wayne-algorithms-exercises","name":"Sedgewick and Wayne Algorithms Exercises","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seed-bench","name":"SEED-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (47 entries), launch posts (3 scores) and system cards (2 scores)."],["scores.models","pass","12 models we track, plus 38 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for LFM2.5-VL-3B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 77.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 9 rows for models we track and 38 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seed-bench-2","name":"SEED-Bench-2","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","5 models we track, plus 26 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Claude 3 Opus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 71.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 5 rows for models we track and 26 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seed-bench-2-plus","name":"SEED-Bench-2-Plus","summary":{"pass":9,"warn":2,"fail":0,"info":1,"text":"9 of 11 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (210 entries) and system cards (3 scores)."],["scores.models","pass","138 models we track, plus 74 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Nemotron Nano 12B v2 VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 73.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on OpenCompass: 137 rows for models we track and 74 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seed-tts-eval","name":"Seed-TTS-eval","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (5 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Ming-3.0-tts-0.5B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seed-vl-internal-benchmark-bytedance","name":"Seed1.5-VL internal benchmark (ByteDance Seed in-house)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Seed1.5-VL. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 59.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seedbench-seed-science","name":"SeedBench (Seed Science)","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for DeepSeek-V3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3 (#141)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seedclawbench-bytedance","name":"SeedClawBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from June 2026, for Doubao Seed 2.1 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 66.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on seed.bytedance.com: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seedkernelbench-bytedance","name":"SeedKernelBench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seeing-what-execution-needs","name":"Seeing What Execution Needs","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","6 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","pass","The best is 42.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"segment-anything-video","name":"Segment Anything Video","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"segment-anything-with-concepts","name":"Segment Anything with Concepts","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on SOTA2: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"metr-ml-engineering-tasks","name":"Select machine learning engineering tasks from METR","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"self-harm-refusals-xai","name":"Self-harm refusals (xAI)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"self-instruct-user-oriented-instructions","name":"Self-Instruct User-Oriented Instructions","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"self-preference-bias-anthropic","name":"Self-preference bias (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gdm-self-proliferation","name":"Self-proliferation challenges (Phuong et al. 2024)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Gemma 2 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 2 27B (#190)."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"semantic-dlp-catch-the-data-leak","name":"Semantic DLP: Catch the Data Leak","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Gemini 2.5 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.5 Flash (#120)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since June 2025."],["import.importer","pass","We read the board on kaggle.com: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"semaplc-function-track","name":"SemaPLC Function Track","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from June 2026, for GLM-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 82.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 35 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"semaplc-project-context","name":"SemaPLC Project-Context Track","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from June 2026, for GLM-5.2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since April 2026."],["import.importer","pass","We read the board on arxiv.org: 28 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"semclinbr","name":"SemClinBr","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"semcomp-bench","name":"SemComp-Bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","2 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Wan2.2-I2V-A14B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on semcomp-bench.github.io: 2 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"semeval-2014-clinical-text","name":"SemEval 2014 Task 7 Analysis of Clinical Text","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"semeval-2015-clinical-text","name":"SemEval 2015 Task 14 Analysis of Clinical Text","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"semeval-2022-task-8-multilingual-news-article-similarity","name":"SemEval 2022 Task 8 multilingual news article similarity","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (127 entries)."],["scores.models","pass","4 models we track, plus 123 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Qwen3-Embedding-0.6B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MTEB: 4 rows for models we track and 123 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"semeval-2023-task-3-multilingual-news","name":"SemEval 2023 Task 3: Genre, Framing and Persuasion Techniques in Online News","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on propaganda.math.unipd.it: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"semeval-2024-task-1-semantic-textual-relatedness","name":"SemEval 2024 Task 1 semantic textual relatedness","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (193 entries)."],["scores.models","pass","4 models we track, plus 189 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on MTEB: 4 rows for models we track and 189 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"semeval-2026-task-4-narrative-similarity","name":"SemEval 2026 Task 4: Narrative Story Similarity and Narrative Representation Learning","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (46 entries)."],["scores.models","pass","46 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on narrative-similarity-task.github.io: 46 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"semeval-2015-task-1","name":"SemEval-2015 Task 1","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"semikongbench-100","name":"SemiKongBench-100","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fab-send-pedagogy-benchmark","name":"SEND Pedagogy Benchmark","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (247 entries)."],["scores.models","pass","230 models we track, plus 17 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 89.5%, past 85% since June 2025."],["import.importer","pass","We read the board on fab-ai.org: 258 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"senior-engineer-benchmark-every","name":"Senior Engineer Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on every.to: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"senior-engineer-benchmark-every-checks","name":"Senior Engineer Benchmark (Every Checks)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 85%, past 85% since September 2026."],["import.importer","pass","We read the board on checks.every.to: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"senior-swe-bench","name":"Senior SWE-Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","23 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 39.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on senior-swe-bench.snorkel.ai: 23 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"seniortalk","name":"SeniorTalk","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","1 model we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for Whisper large-v3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sentences-involving-compositional-knowledge","name":"Sentences Involving Compositional Knowledge","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sentinelbench","name":"SentinelBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sequential-question-answering","name":"Sequential Question Answering","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sequitur-lab-automation","name":"Sequitur Research Lab Automation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"settlerbench-season-one","name":"SettlerBench Season One","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","21 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on synopticonresearch.com: 24 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sevenllm","name":"SEvenLLM","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sexually-explicit-content-safety-eval-google","name":"Sexually Explicit Content Safety","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 97.6%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sf20k-competition-2025","name":"SF20K Competition 2025","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sf20k-test","name":"SF20K-Test","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","7 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Pixtral 12B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ridouaneg.github.io: 20 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sfi-bench","name":"SFI-Bench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sg-legalclarify","name":"SG-LegalClarify","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","16 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 77.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 32 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sgi-bench","name":"SGI-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries) and launch posts (1 score)."],["scores.models","pass","16 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Intern-S2-397B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","pass","The best is 51.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on internscience.github.io: 15 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sgp-bench","name":"SGP-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","24 models we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for InternVL3.5-241B-A28B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","pass","The best is 70.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sgp-bench.github.io: 23 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"shade-arena","name":"SHADE-Arena","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (16 scores)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 26.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"shadowbench","name":"ShadowBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 18 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"share-clef-2013-disorder-recognition-normalization","name":"ShARe/CLEF eHealth 2013 Task 1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sheetbench-hud","name":"SheetBench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 64.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on fireworks.ai: 9 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sherlock-holmes-qa-gutenberg","name":"Sherlock Holmes Q&A","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"shop-assistant-honesty-bench-ua-ru","name":"Shop Assistant Honesty Bench (UA/RU)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Sonnet 5.5 (#9)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since August 2025."],["import.importer","pass","We read the board on kaggle.com: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"shopping-mmlu","name":"Shopping MMLU","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","14 models we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Llama 3-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-70B (#182)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 14 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"short-form-bench","name":"Short Form Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on usenaive.ai: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"short-horizon-computational-biology-tasks-anthropic","name":"Short-horizon computational biology tasks","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"shp2-protein-function-prediction-openai","name":"SHP2 Protein Function Prediction","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (10 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"shrec-2026-human-perceived-visual-complexity","name":"SHREC 2026 Human Perceived Visual Complexity of 3D Shapes","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"shroom-cap","name":"SHROOM-CAP","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for gpt-oss-20b."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is gpt-oss-20b (#143)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"shroom-visions-2026","name":"SHROOM-Visions 2026","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on helsinki-nlp.github.io: 11 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sib-200","name":"SIB-200","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in June 2023, for GPT-3.5 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","pass","The best is 48.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sick-nl","name":"SICK-NL","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sifo","name":"SIFo","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Qwen3-VL-235B-A22B. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 77.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sight-meituan","name":"SIGHT (Search and Intent Gauge for Hospitality and Tourism)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sighum-sanskrit-word-segmentation","name":"SIGHUM Sanskrit Word Segmentation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sigmorphon-2023-interlinear-glossing","name":"SIGMORPHON 2023 Shared Task on Interlinear Glossing","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 12 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"signbd-word","name":"SignBD-Word","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"silent-copying-of-answers-anthropic","name":"Silent copying of answers (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"silent-exit","name":"Silent Exit","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"silverquillm-bench-hob-medium","name":"SilverquiLLM-bench hob-medium","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 96.9%, past 85% since September 2026."],["import.importer","pass","We read the board on snowfoxbuilds.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sim-vail","name":"SIM-VAIL","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on mheval.slingshotai.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simbench-pychrono","name":"SimBench (PyChrono)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on uwsbel.github.io: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simple-cooccurrence-bias","name":"Simple Cooccurrence Bias","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simplebench","name":"SimpleBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (114 entries), boards Epoch AI copied (9 models) and claims Epoch AI collected (18 models)."],["scores.models","pass","111 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 94.9%, past 85% since September 2026."],["import.importer","pass","We read the board on simple-bench.com: 113 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simpleqa","name":"SimpleQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries), launch posts (42 scores) and system cards (40 scores)."],["scores.models","pass","69 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Granite 4.1 30B. That’s expected for a test replaced by SimpleQA Verified."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 97.1%, past 85% since January 2025."],["import.importer","pass","We read the board on github.com: 22 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 7 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simpleqa-hallucination-rate","name":"SimpleQA Hallucination Rate","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries), launch posts (1 score) and system cards (12 scores)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simpleqa-verified","name":"SimpleQA Verified","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (57 entries), Epoch AI’s runs (96 models), launch posts (13 scores) and system cards (17 scores)."],["scores.models","pass","133 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 94%, past 85% since November 2026."],["import.importer","pass","We read the board on kaggle.com: 58 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 10 pairs differs by more than 10 points, the largest for DeepSeek-V4-Pro (+11 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simplerenv","name":"SimplerEnv","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores) and system cards (1 score)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from November 2026, for Praxis-1."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simplesafetytests","name":"SimpleSafetyTests","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (81 entries) and system cards (1 score)."],["scores.models","pass","76 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Gemini 3 Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since June 2023."],["import.importer","pass","We read the board on Stanford HELM: 81 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simplevqa","name":"SimpleVQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (21 entries), launch posts (28 scores) and system cards (9 scores)."],["scores.models","pass","44 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 81.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 14 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simready-text-to-cad","name":"SimReady Text-to-CAD","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","info","Saturated. The best is 96.2%, past 85% since September 2026."],["import.importer","pass","We read the board on github.com: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simreadybench-nv100","name":"SimReadyBench NV100","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2026."],["import.importer","pass","We read the board on huggingface.co: 13 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simreal-ml-benchmark","name":"Simreal ML Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simrepairbench","name":"SimRepairBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simulatorarena","name":"SimulatorArena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on simulatorarena.ai: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simulcost","name":"SimulCost","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 80.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"simuverity","name":"SimuVerity","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on simuverity.github.io: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"singapore-factuality","name":"Singapore Factuality","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"singapore-pofma-statements","name":"Singapore POFMA Statements","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"singapore-safety-questions","name":"Singapore Safety Questions","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"singapore-undesirable-content","name":"Singapore Undesirable Content","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"single-turn-benign-request-evaluations-anthropic","name":"Single-turn benign request evaluations (Anthropic)","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (19 scores)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"election-integrity-single-turn-2026-anthropic","name":"Single-turn election integrity (Anthropic, 2026)","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"single-turn-harmful-request-evaluations-anthropic","name":"Single-turn harmful request evaluations (Anthropic)","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (19 scores)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 99.8%, past 85% since May 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"singmos","name":"SingMOS","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sinhala-language-proficiency-linguistic-mastery","name":"Sinhala Language Proficiency & Linguistic Mastery","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"siqa","name":"SIQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries), launch posts (7 scores) and system cards (23 scores)."],["scores.models","pass","24 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","pass","The best is 80.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"site-bench","name":"SITE","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (9 entries) and launch posts (1 score)."],["scores.models","pass","7 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for SenseNova-SI-1.3-InternVL3-8B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 47.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on wenqi-wang20.github.io: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"situational-awareness-dataset-sad","name":"Situational Awareness Dataset (SAD)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","17 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Muse Spark."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","pass","The best is 61.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on situational-awareness-dataset.org: 34 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"skatebench-2","name":"SkateBench v2","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","21 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 Pro (#19)."],["scores.headroom","info","Saturated. The best is 96.9%, past 85% since February 2026."],["import.importer","pass","We read the board on skatebench.t3.gg: 28 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"skill-reward-hacking","name":"Skill Reward Hacking","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for MiMo-V2.5-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GLM-5.1 (#69)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on app.primeintellect.ai: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"skillinject","name":"SkillInject","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"skillsbench","name":"SkillsBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries), launch posts (13 scores) and system cards (2 scores)."],["scores.models","pass","23 models we track, plus 16 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for LongCat-2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 70.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on skillsbench.ai: 25 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"skillspan","name":"SkillSpan","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"skillspan-esco","name":"SkillSpan-ESCO","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on ugentt2k.github.io: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sklep","name":"skLEP","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","14 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 14 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"slake","name":"SLAKE","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (10 scores) and system cards (1 score)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","info","Saturated. The best is 90.8%, past 85% since August 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"slate","name":"SLATE","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 5.46%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sts-bench-alavico","name":"Slay the Spire Bench (alavico)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","12 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on alavico.github.io: 25 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sldbench","name":"SLDBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"slide-generation-human-eval-zai","name":"Slide generation human evaluation (Z.ai)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for GLM-5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GLM-5 (#89)."],["scores.headroom","pass","The best is 67.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"slidevqa","name":"SlideVQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on SOTA2: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"slip-zero-shot-transfer-suite","name":"SLIP zero-shot transfer suite","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"slitherlink-env","name":"Slitherlink (savi)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"slomo-audio-description-2026","name":"SLoMO Audio Description Competition 2026","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on slomo-workshop.github.io: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"slop-arena","name":"Slop Arena","summary":{"pass":1,"warn":3,"fail":1,"info":5,"text":"1 of 5 checks pass, 3 warnings, 1 problem"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","fail","Unverified. We haven’t checked it against a primary source yet."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on sloparena.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"slop-score","name":"Slop Score","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","17 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 4.5 (#81)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eqbench.com: 18 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"slopcodebench","name":"SlopCodeBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","22 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 32.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on scbench.ai: 32 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"slphelm-ultrasuiteplus","name":"SLPHelm UltraSuitePlus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"slr-bench","name":"SLR-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries)."],["scores.models","pass","32 models we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 81.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 43 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"slurp","name":"SLURP","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"small-model-agent-bench","name":"Small Model Agent Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Saturated. The best is 95.7%, past 85% since July 2025."],["import.importer","pass","We read the board on kaggle.com: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rtlrewriter-small","name":"Small Rewriter Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for GPT-4V."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smartplay","name":"SmartPlay","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Llama 2-13B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smb-bench","name":"SMB Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smdd-bench","name":"SMDD-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for MiniMax-M2.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 40.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on smddbench.com: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smebench","name":"SMEBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smile-video-laugh-reasoning","name":"SMILE Video Laugh Reasoning","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smm4h-2025-food-recalls-and-outbreaks","name":"SMM4H 2025 Food Recalls and Foodborne Disease Outbreaks","summary":{"pass":6,"warn":2,"fail":0,"info":4,"text":"6 of 8 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for o3-mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3-mini (#124)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on workshop-proceedings.icwsm.org: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smm4h-2025-shingles-vaccine-reactions","name":"SMM4H 2025 Shingles Vaccine Reactions","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codalab.lisn.upsaclay.fr: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smm4h-insomnia-2025","name":"SMM4H-HeaRD 2025 Detection of Insomnia in Clinical Notes","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","3 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 22 months ago, in January 2025, for o3-mini. That’s expected for a test replaced by SMM4H-HeaRD 2026 Detection of Insomnia in Clinical Notes."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3-mini (#124)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on workshop-proceedings.icwsm.org: 3 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smm4h-multilingual-ade-2025","name":"SMM4H-HeaRD 2025 Multilingual Adverse Drug Event Detection","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codalab.lisn.upsaclay.fr: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smm4h-insomnia-2026","name":"SMM4H-HeaRD 2026 Detection of Insomnia in Clinical Notes","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codabench.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smm4h-heard-2026-fluve","name":"SMM4H-HeaRD 2026 Flu Vaccine Effectiveness","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codabench.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smm4h-heard-2026-dial2note","name":"SMM4H-HeaRD 2026 MedSynth Dialogue-to-Note","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codabench.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smm4h-multilingual-ade-2026","name":"SMM4H-HeaRD 2026 Multilingual Adverse Drug Event Detection","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codabench.org: 12 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smm4h-heard-2026-tnm","name":"SMM4H-HeaRD 2026 Pathology Report Staging","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Mini (#74)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smolinstruct","name":"SMolInstruct","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Intern-S1-Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 74.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sms-spam-collection","name":"SMS Spam Collection","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smt-2025","name":"SMT 2025","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries)."],["scores.models","pass","40 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for QED-Nano. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Saturated. The best is 93.4%, past 85% since April 2025."],["import.importer","pass","We read the board on MathArena: 43 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snakebench","name":"SnakeBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (306 entries)."],["scores.models","pass","201 models we track, plus 105 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on snakebench.com: 218 rows for models we track and 105 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sneaky-sandbox-escape-openai","name":"Sneaky Sandbox Escape","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkel-agentic-coding","name":"Snorkel Agentic Coding","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. That’s expected for a test replaced by Snorkel Agentic Coding 2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 65.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snorkel.ai: 11 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkel-agentic-coding-2","name":"Snorkel Agentic Coding 2.0","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 47.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snorkel.ai: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swim-snorkel","name":"Snorkel Working Memory Test","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in April 2024, for GPT-4 Turbo. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 Turbo (#172)."],["scores.headroom","info","Saturated. The best is 87.6%, past 85% since February 2024."],["import.importer","pass","We read the board on github.com: 7 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkelfinance","name":"SnorkelFinance","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","22 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Gemini 3 Pro. That’s expected for a test replaced by SnorkelFinance 2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 81%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snorkel.ai: 22 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkelfinance-2","name":"SnorkelFinance 2.0","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 25.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snorkel.ai: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkelgraph","name":"SnorkelGraph","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","24 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 84.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snorkel.ai: 24 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkellegal","name":"SnorkelLegal","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 46%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snorkel.ai: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkelmanufacturing","name":"SnorkelManufacturing","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 27.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snorkel.ai: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkelrevops","name":"SnorkelRevOps","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 18.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snorkel.ai: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkelsequences","name":"SnorkelSequences","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","25 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 77.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snorkel.ai: 25 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkelspatial","name":"SnorkelSpatial","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","27 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 99%, past 85% since March 2026."],["import.importer","pass","We read the board on snorkel.ai: 27 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkelunderwrite","name":"SnorkelUnderwrite","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","27 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4. That’s expected for a test replaced by SnorkelUnderwrite 2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 91%, past 85% since August 2025."],["import.importer","pass","We read the board on snorkel.ai: 27 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkelunderwrite-2","name":"SnorkelUnderwrite 2.0","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 30.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snorkel.ai: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkelwordle","name":"SnorkelWordle","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","19 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Gemini 3 Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Saturated. The best is 94%, past 85% since January 2025."],["import.importer","pass","We read the board on snorkel.ai: 19 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-extract-11-datasets-snowflake","name":"Snowflake AI_EXTRACT Document Extraction Evaluation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on snowflake.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"arxiv-papers-analytical-search-snowflake","name":"Snowflake Analytical Search: ArXiv Papers","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"enterprise-intelligence-snowflake","name":"Snowflake Arctic Enterprise Intelligence","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"finance-rag-snowflake","name":"Snowflake Financial Filings RAG Evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"parse-document-financial-tables-snowflake","name":"Snowflake Financial PDF Table Parsing","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"multimodal-pdf-retrieval-snowflake","name":"Snowflake Multimodal PDF Retrieval Evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snowflake-text-to-sql-hard","name":"Snowflake Text-to-SQL HARD","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","12 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Claude Opus 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 47.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snowflake.com: 12 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"so-arena","name":"SO-Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","2 models we track, plus 20 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-6 Sol (#13)."],["scores.headroom","pass","The best is 82.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on lichenlillc.github.io: 2 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"so-bench","name":"SO-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"soccernet-caption","name":"SoccerNet-Caption","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on soccer-net.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"social-science-experiment-treatment-effect-archive","name":"Social Science Experiment Treatment Effect Archive","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"social-signal-bench","name":"Social Signal Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"socialmaze","name":"SocialMaze","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sociohack","name":"SocioHack","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"socrates","name":"SoCRATES","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sofc-exp","name":"SOFC-Exp","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 11 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"soft-bias-xai","name":"Soft Bias (Internal)","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Grok 4 Fast. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4 (#86)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sokobench","name":"SokoBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sol-execbench","name":"SOL-ExecBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on research.nvidia.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"solana-machine-trading","name":"Solana Machine Trading","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","6 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 6 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"solidworks-bench","name":"SolidWorks Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on markovstudios.com: 9 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sol-env","name":"Solubility Optimization (anindya)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"solveedit","name":"SolveEdit","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","9 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Flash Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 55.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on wenjieshu.github.io: 9 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"something-something-v2","name":"Something-Something V2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (61 entries)."],["scores.models","pass","1 model we track, plus 60 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for V-JEPA 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 74%, under 85%, so it still separates models."],["import.importer","pass","We read the board on SOTA2 Research: 1 row for models we track and 60 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"norwegian-snomed-ct-term-recognition-sonetra","name":"Sonetra Norwegian SNOMED CT Term Recognition","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for gpt-4o-transcribe."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 66.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sonetra.no: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"songbench","name":"SongBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"songeval","name":"SongEval","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"soohak","name":"Soohak","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sop-bench-amazon","name":"SOP-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sorry-bench","name":"SORRY-Bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (56 entries)."],["scores.models","pass","28 models we track, plus 28 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for o1-mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-mini (#142)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on sorry-bench.github.io: 28 rows for models we track and 28 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sosbench","name":"SoSBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (29 entries)."],["scores.models","pass","23 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 26 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"soundingearth-mteb","name":"SoundingEarth audio and image retrieval (MTEB)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"south-asian-music-understanding-kader","name":"South Asian Music Understanding","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (33 entries)."],["scores.models","pass","23 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Gemma 3 1B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Phi-4 (#160)."],["scores.headroom","pass","The best is 48.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 23 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spa-bench","name":"SPA-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","1 model we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in December 2023, for CogAgent."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 1.35%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"space-apple","name":"SPACE","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"space-10","name":"SpaCE-10","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (46 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","36 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Qwen3-VL-235B-A22B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 55%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 36 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spacechem-bench-wincon","name":"SpaceChem Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on spacechem.wincon.dev: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spacedg-bench","name":"SpaceDG-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (27 entries)."],["scores.models","pass","20 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen 3.6 35B-A3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 56.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 22 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spacexai-internal-r-and-d-evals","name":"SpaceXAI internal R&D evals","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mts-eval-xai","name":"SpaceXAI MTS Eval","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","pass","The best is 61.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spair-71k","name":"SPair-71k","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on SOTA2: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spar-bench","name":"SPAR-Bench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 64.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sparse-fourier-recovery","name":"Sparse Fourier Recovery","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spartqa","name":"SpartQA","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatial-agent-index","name":"Spatial Agent Index v1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatial-agent-index-2","name":"Spatial Agent Index v2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (48 entries)."],["scores.models","pass","47 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on axisspatial.com: 91 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatial-intelligence-index","name":"Spatial Intelligence Index v1","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatial-intelligence-index-2","name":"Spatial Intelligence Index v2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries)."],["scores.models","pass","39 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on axisspatial.com: 53 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatial-pathfinding-reasoning-challenge","name":"Spatial Pathfinding Reasoning Challenge","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","20 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","pass","The best is 83.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sparc.gipplab.org: 23 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatial-reasoning-rotating-ascii-art","name":"Spatial Reasoning: Rotating ASCII Art","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Terra (#18)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatial-gym","name":"Spatial-Gym","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatialbench-latchbio","name":"SpatialBench (LatchBio)","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries) and system cards (7 scores)."],["scores.models","pass","21 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 77.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 19 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatialbench-latchbio-original","name":"SpatialBench (original study)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 38.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatialbench-ropedia","name":"SpatialBench (Ropedia)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatialbench-long","name":"SpatialBench-Long","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 40.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 45 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatialbench-long-original","name":"SpatialBench-Long (original study)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","9 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Gemini 3.5 Flash. That’s expected for a test replaced by SpatialBench-Long."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 11.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatialeval-maze-nav","name":"SpatialEval Maze-Nav","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries) and system cards (2 scores)."],["scores.models","pass","15 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Phi-4-Reasoning-plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 55.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 27 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatialeval-spatial-map","name":"SpatialEval Spatial-Map","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries) and system cards (2 scores)."],["scores.models","pass","15 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Phi-4-Reasoning-plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 73.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 27 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatialgeneval","name":"SpatialGenEval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatialsense-vqa","name":"SpatialSense VQA","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 20 months ago, in March 2025, for Gemma 3 12B. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","pass","The best is 60%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatialtree","name":"SpatialTree","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 58.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatialviz-bench","name":"SpatialViz-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries), system cards (1 score) and claims Epoch AI collected (8 models)."],["scores.models","pass","22 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for STEP3-VL-10B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 45.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on wangst0181.github.io: 31 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatialworld","name":"SpatialWorld","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","11 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 14.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 11 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatialworld-digital","name":"SpatialWorld Digital","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","11 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 39%, under 85%, so it still separates models."],["import.importer","pass","We read the board on spatial-world.github.io: 11 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spatialworld-physical","name":"SpatialWorld Physical","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","11 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 14.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on spatial-world.github.io: 11 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spbench","name":"SPBench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 63.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speaker-identification-openai","name":"Speaker identification","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spec-gaming-review","name":"Spec Gaming Review","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spec2control","name":"Spec2Control","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"specdecode-rl","name":"SpecDecode RL","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speech-agent-arena","name":"Speech Agent Arena","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Grok Voice 3.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speecharena","name":"Speech Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","1 model we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for GPT-4o Mini TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on speecharena.org: 1 row for models we track and 15 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speech-robust-bench","name":"Speech Robust Bench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speech-to-speech-index","name":"Speech-to-Speech Index","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Grok Voice 3.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speecheditbench","name":"SpeechEditBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","2 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for gpt-realtime. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 96.7%, past 85% since August 2025."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speechio-asr-leaderboard","name":"SpeechIO ASR leaderboard","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speechmmlu","name":"SpeechMMLU","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speechrole","name":"SpeechRole","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (1 score)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen3.5-Omni-Plus."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speed-bench","name":"SPEED-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries) and system cards (2 scores)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Nemotron 3 Ultra (#85)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 19 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speedrunbench","name":"SpeedrunBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on patronus.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speko-llm-benchmark-v3","name":"Speko LLM Benchmark for Voice Agents v3","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","26 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Terra (#18)."],["scores.headroom","info","Saturated. The best is 94.7%, past 85% since April 2025."],["import.importer","pass","We read the board on speko.com: 29 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speko-s2s-10-turn","name":"Speko Speech-to-Speech Benchmark (10-turn call)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","6 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-Realtime-2.1."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 1108%, past 85% since August 2025."],["import.importer","pass","We read the board on speko.com: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speko-s2s-20-turn","name":"Speko Speech-to-Speech Benchmark (20-turn concierge call)","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for GPT-Realtime-2.1 Mini."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on speko.com: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"speko-s2s-six-scenarios","name":"Speko Speech-to-Speech Benchmark (six concierge scenarios)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","11 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Grok Voice 3.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on speko.com: 11 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spgispeech","name":"SPGISpeech","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (68 entries)."],["scores.models","pass","9 models we track, plus 59 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for NVIDIA Nemotron 3.5 Transcribe 30B-A4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on Open ASR Leaderboard: 9 rows for models we track and 59 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spi-security-patch-identification","name":"SPI (security patch identification)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Gemma 2 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 2 27B (#190)."],["scores.headroom","pass","The best is 53%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spider","name":"Spider","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries), launch posts (1 score) and system cards (3 scores)."],["scores.models","pass","6 models we track, plus 27 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Qwen3-Coder-Next. That’s expected for a test replaced by Spider 2.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 83.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on yale-lily.github.io: 3 rows for models we track and 27 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spider-2","name":"Spider 2.0","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (50 entries) and launch posts (1 score)."],["scores.models","pass","24 models we track, plus 26 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Saturated. The best is 94.2%, past 85% since September 2025."],["import.importer","pass","We read the board on spider2-sql.github.io: 32 rows for models we track and 26 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spider2-aifunc","name":"Spider 2.0-AIFunc","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 70.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spider2-v","name":"Spider2-V","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 14%, under 85%, so it still separates models."],["import.importer","pass","We read the board on spider2-v.github.io: 5 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spiral-bench","name":"Spiral-Bench v1.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5. That’s expected for a test replaced by Spiral-Bench v1.2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eqbench.com: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spiral-bench-1-2","name":"Spiral-Bench v1.2","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","21 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eqbench.com: 22 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spirebench","name":"SpireBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","An importer exists, but it hasn’t read the board yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spirebench-hermesbridge-trial-v0","name":"SpireBench (HermesBridge trial-v0)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on spirebench.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spireclimb","name":"SpireClimb","summary":{"pass":8,"warn":1,"fail":0,"info":1,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from Epoch AI’s runs (12 models)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","8 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 41.3%, under 85%, so it still separates models."],["import.importer","warn","The record links a board on epoch.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spireclimb-sts1-a20-heart","name":"SpireClimb StS1 A20 Heart","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries) and Epoch AI’s runs (12 models)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","8 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 33.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on epoch.ai: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spireclimb-sts2-a10","name":"SpireClimb StS2 A10","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries) and Epoch AI’s runs (12 models)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","8 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 49.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on epoch.ai: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"splatbench","name":"SplatBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spm-bench","name":"SPM-Bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 17 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sportseval-english-icelandic","name":"SportsEval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","13 models we track, plus 21 other entries on its leaderboard."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Saturated. The best is 90%, past 85% since June 2025."],["import.importer","pass","We read the board on github.com: 13 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spot","name":"SPOT","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for o3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 6.10%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spot5-daily-photograph-scheduling","name":"SPOT5 daily photograph scheduling","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spotsoundbench","name":"SpotSoundBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","4 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 67.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on loiesun.github.io: 3 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spreadsheetbench","name":"SpreadsheetBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries), launch posts (5 scores) and system cards (2 scores)."],["scores.models","pass","7 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.7-Max (#49)."],["scores.headroom","info","Saturated. The best is 89.3%, past 85% since May 2026."],["import.importer","pass","We read the board on spreadsheetbench.github.io: 2 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 1 pair, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spreadsheetbench-2","name":"SpreadsheetBench 2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries), launch posts (3 scores) and system cards (1 score)."],["scores.models","pass","11 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for LongCat-2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 34.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on spreadsheetbench.github.io: 8 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spreadsheetbench-verified","name":"SpreadsheetBench Verified","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","3 models we track, plus 17 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is MiniMax-M3 (#80)."],["scores.headroom","pass","The best is 82.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on spreadsheetbench.github.io: 1 row for models we track and 17 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spy-pii-detection","name":"SPY","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"spyfall-study","name":"Spyfall study","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-8B (#154)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sqad","name":"SQAD","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sql-reality-check","name":"SQL Reality Check","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since April 2026."],["import.importer","pass","We read the board on kaggle.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sql-api-bench-ibm","name":"SQL-API-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"squad","name":"SQuAD","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (124 entries), launch posts (1 score) and system cards (5 scores)."],["scores.models","pass","5 models we track, plus 124 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in July 2024, for Llama 3.1-405B. That’s expected for a test replaced by SQuAD v2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Saturated. The best is 89.3%, past 85% since April 2024."],["import.importer","pass","We read the board on rajpurkar.github.io: 124 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"squad-v2","name":"SQuAD v2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (174 entries), launch posts (1 score) and system cards (7 scores)."],["scores.models","pass","8 models we track, plus 174 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for Llama 3.1-405B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","pass","The best is 81.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on rajpurkar.github.io: 174 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"squadstack-conversational-llm-benchmark","name":"SquadStack Conversational LLM Benchmark for Voice Agents","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","9 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Saturated. The best is 92.5%, past 85% since April 2026."],["import.importer","pass","We read the board on huggingface.co: 9 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"squality","name":"SQuALITY","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (6 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 20 months ago, in March 2025, for Phi-4 Mini. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 24.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sre-bench","name":"SRE-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (37 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","36 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Small."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 92.8%, past 85% since September 2026."],["import.importer","pass","We read the board on vals.ai: 35 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 1 pair, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sregym-env","name":"Sregym env","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sregym-lite","name":"SREGym-Lite","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Sonnet 5.5 (#9)."],["scores.headroom","pass","The best is 81.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dri-factum-sri-lanka-local-elections-2025","name":"Sri Lankan local election chatbot audit","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on democracyreporting.s3.eu-central-1.amazonaws.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ssh-stress-testing-evaluation-anthropic","name":"SSH stress-testing evaluation (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Saturated. The best is 95%, past 85% since February 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"st-vqa","name":"ST-VQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stabletoolbench","name":"StableToolBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for GPT-4o mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o mini (#178)."],["scores.headroom","pass","The best is 26.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stack-eol-audit-geol-cli-vs-api","name":"Stack EOL Audit: geol CLI vs API","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Mini (#74)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stage-cross-scene-narrative-evolution-reasoning","name":"STAGE Cross-Scene Narrative Evolution Reasoning","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 65.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 36 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stale-memory","name":"STALE","summary":{"pass":7,"warn":3,"fail":0,"info":2,"text":"7 of 10 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","10 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Nano."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 55.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 10 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"standard-jailbreaks-xai","name":"Standard jailbreaks (xAI)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"standard-refusal-evals-openai","name":"Standard Refusal Evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (8 scores)."],["scores.models","pass","8 models we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5 mini. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1 (#118)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since August 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stanford-cars","name":"Stanford Cars","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stanford-online-products","name":"Stanford Online Products","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","23 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on Open Metric Learning: 23 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"star-situated-video-reasoning","name":"STAR situated video reasoning","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stare-spatial","name":"STARE","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for STEP3-VL-10B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 61.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stark","name":"STaRK","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 12 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"starskirmish-bench","name":"StarSkirmish Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder Full."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 57.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on starskirmish.com: 9 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"state-bench","name":"STATE-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GPT-5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 58.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on microsoft.github.io: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"state-bench-main-0-7-0","name":"STATE-Bench Main 0.7.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro. That’s expected for a test replaced by STATE-Bench."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 55.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on microsoft.github.io: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"statebench","name":"StateBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"statebench-video-continuation","name":"StateBench (video continuation)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Wan2.2-I2V-A14B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"states-united-voter-questions-2026","name":"States United voter questions","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"static-jailbreak-evals-openai","name":"Static Jailbreak Evaluations","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stationerybench-robocurve","name":"StationeryBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on openai.robocurve.org: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stealth-and-situational-awareness-evaluations","name":"Stealth and Situational Awareness evaluations","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 93.3%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"steerbench-spoken","name":"SteerBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stella-bench-atella","name":"STELLA-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Sonnet 5.5 (#9)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on leaderboard.atella.ai: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stem-professor-bench-anthropic","name":"STEM Professor Bench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 61.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"step-race-lech-mazur","name":"Step Race","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (68 entries)."],["scores.models","pass","59 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Mistral Large 3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 66 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"step-caption","name":"Step-Caption","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"step-du","name":"Step-DU","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"step-spqa","name":"Step-SPQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"step-swe-bench","name":"Step-SWE-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Step 3.7 Flash. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 67.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stepcodebench","name":"StepCodeBench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and launch posts (1 score)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for Step 5 Preview."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 63.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on stepfun.com: 5 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stepeval-audio-360","name":"StepEval-Audio-360","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stepup-gym-attendance-forecasts","name":"StepUp gym attendance forecasts","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stereoset","name":"StereoSet","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 11 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stereotales","name":"StereoTales","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stick-to-your-role","name":"Stick to Your Role!","summary":{"pass":6,"warn":3,"fail":0,"info":3,"text":"6 of 9 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","17 models we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-235B-A22B (#131)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 17 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stl-10","name":"STL-10","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stockbench","name":"StockBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Kimi K2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on stockbench.github.io: 13 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"storal","name":"STORAL","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"storebench","name":"StoreBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","6 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on afterquery.com: 13 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"storycloze-2018","name":"Story Cloze Winter 2018","summary":{"pass":7,"warn":0,"fail":1,"info":4,"text":"7 of 8 checks pass, 1 problem"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on competitions.codalab.org: 11 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","fail","The read on November 19th, 2026 failed with “fetch failed”. Pages show the rows from November 18th, 2026."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"storycloze","name":"StoryCloze","summary":{"pass":7,"warn":0,"fail":1,"info":4,"text":"7 of 8 checks pass, 1 problem"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 15 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in May 2023, for PaLM 2-L. That’s expected for a test replaced by Story Cloze Winter 2018."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is PaLM 2-L (#198)."],["scores.headroom","info","Saturated. The best is 87.4%, past 85% since May 2023."],["import.importer","pass","We read the board on competitions.codalab.org: 15 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","fail","The read on November 19th, 2026 failed with “fetch failed”. Pages show the rows from November 18th, 2026."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stratabench-land-v1","name":"StrataBench Land v1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Luna (#36)."],["scores.headroom","pass","The best is 0.94%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 25 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stratabench-land-v1-classify","name":"StrataBench Land v1 Classify","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Luna (#36)."],["scores.headroom","info","Saturated. The best is 93.9%, past 85% since December 2024."],["import.importer","pass","We read the board on huggingface.co: 25 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"stratabench-land-v1-extract","name":"StrataBench Land v1 Extract","summary":{"pass":7,"warn":1,"fail":1,"info":3,"text":"7 of 9 checks pass, 1 warning, 1 problem"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Luna (#36)."],["scores.headroom","info","Saturated. The best is 88.1%, past 85% since December 2024."],["import.importer","pass","We read the board on huggingface.co: 10 rows for models we track at the last read."],["import.read","warn","November 8th, 2026, 13 days before the data’s date."],["import.failed","fail","The read on November 19th, 2026 failed with “401 https://huggingface.co/datasets/Collide-ai/stratabench-land-v1/raw/main/README.md”. Pages show the rows from November 8th, 2026."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"strategy-bench","name":"Strategy Bench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","warn","The record links a board on strategy.freysa.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"strategyqa","name":"StrategyQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","2 models we track, plus 8 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Cohere Command A. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is PaLM 2-L (#198)."],["scores.headroom","info","Saturated. The best is 90.4%, past 85% since May 2023."],["import.importer","pass","We read the board on github.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"streambench-streamchat","name":"StreamBench (StreamChat)","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","6 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Seed1.5-VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o mini (#178)."],["scores.headroom","pass","The best is 72.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"streameval","name":"StreamEval","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on streameval.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"streamingbench","name":"StreamingBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","8 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 84.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"streamingqa","name":"StreamingQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"strongreject","name":"StrongREJECT","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (4 scores) and system cards (19 scores)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Saturated. The best is 98.6%, past 85% since February 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"strongreject-v2-meta","name":"StrongREJECT v2 (Meta)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (3 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"structeval","name":"StructEval","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","29 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","pass","The best is 81.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tiger-ai-lab.github.io: 28 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"structural-biology-anthropic","name":"Structural Biology (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","pass","The best is 79%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"structured-expert-probing-chem-bio-openai","name":"Structured expert probing campaign: chem-bio novel design","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 23 months ago, in December 2024, for o1. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"structured-expert-probing-rn-openai","name":"Structured expert probing campaign: radiological and nuclear","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 22 months ago, in January 2025, for o3-mini. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"structured-output-benchmark","name":"Structured Output Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (30 entries)."],["scores.models","pass","22 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 87%, past 85% since April 2025."],["import.importer","pass","We read the board on interfaze.ai: 22 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"structured-translation-cohere","name":"Structured Translation (Cohere)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for North Small Translate 1.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 93.7%, past 85% since September 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swde","name":"Structured Web Data Extraction","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"student-work-grading","name":"Student-Work Grading","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Haiku 5.5 (#20)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"studentbench","name":"StudentBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","11 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on joinhandshake.com: 12 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"studyarena-audio","name":"StudyArena Audio","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","3 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemini 3.1 Flash TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on studyarena.com: 3 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"studyarena-image","name":"StudyArena Image","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (46 entries)."],["scores.models","pass","29 models we track, plus 17 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-Image-2.6 Flare."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on studyarena.com: 29 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"studyarena-text","name":"StudyArena Text","summary":{"pass":9,"warn":0,"fail":1,"info":2,"text":"9 of 10 checks pass, 1 problem"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (106 entries)."],["scores.models","pass","86 models we track, plus 20 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on studyarena.com: 276 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","fail","The read on November 20th, 2026 failed with “500 https://studyarena.com/leaderboard”. Pages show the rows from November 18th, 2026."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"styleinstructcaps","name":"StyleInstructCaps","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"subversion-strategy-eval","name":"Subversion Strategy Eval","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Claude Opus 4.5."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"success-detection-google","name":"Success detection (Google DeepMind)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemini Robotics ER 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 90%, past 85% since April 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sudo-l7","name":"sudo L7","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries) and launch posts (1 score)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 52.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on surgehq.ai: 19 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sudoku-bench","name":"Sudoku-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for LongCat-Flash-Thinking."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 56%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 22 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sugarcrepe","name":"SugarCrepe","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (58 entries)."],["scores.models","pass","2 models we track, plus 56 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in July 2024, for InternVL2-8B. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 95.2%, past 85% since June 2024."],["import.importer","pass","We read the board on SOTA2: 2 rows for models we track and 56 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sugars-kitchen-ai-cooking-battle-17","name":"Sugar’s Kitchen AI Cooking Battle 17","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on note.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"suicide-self-harm-eval-google","name":"Suicide & Self-Harm evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 96%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"suicide-and-self-harm-evaluations-anthropic","name":"Suicide and self-harm evaluations (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"suicide-memory-bench","name":"Suicide Memory Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on prosaicintelligence.ai: 16 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sumi-ja-pii-corpus","name":"Sumi Japanese PII Corpus","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"summ-it-coreference","name":"Summ-it Coreference","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"summhay","name":"Summary of a Haystack","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","3 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in March 2024, for Claude 3 Haiku."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","pass","The best is 42.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 21 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"summscreenfd","name":"SummScreenFD","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on SOTA2 Research: 12 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sumo-web-claims-summarization","name":"SUMO Web Claims Summarization","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 10 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sun397","name":"SUN397","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sunbird-african-language-technology","name":"Sunbird African Language Technology","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sunrgbd","name":"SUNRGBD","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (7 scores) and system cards (3 scores)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Qwen3.5-2B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5 397B-A17B (#78)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"supabase-evals","name":"Supabase Evals","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 92.8%, past 85% since July 2026."],["import.importer","pass","We read the board on supabase.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superb","name":"SUPERB","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superbbenchmark.github.io: 19 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superchem","name":"SUPERChem","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries), launch posts (4 scores) and system cards (2 scores)."],["scores.models","pass","11 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 66.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on superchem.pku.edu.cn: 16 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue","name":"SuperCLUE","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","18 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Qwen4-Max."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 18 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-arena-image-editing","name":"SuperCLUE Arena Image Editing","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","15 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 15 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-arena-image-to-video","name":"SuperCLUE Arena Image to Video","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","25 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 25 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-arena-reference-to-video","name":"SuperCLUE Arena Reference to Video","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 12 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-arena-text-to-image","name":"SuperCLUE Arena Text to Image","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","17 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for Claude 2.1."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 17 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-arena-text-to-speech","name":"SuperCLUE Arena Text to Speech","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","3 models we track, plus 11 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 3 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-arena-text-to-video","name":"SuperCLUE Arena Text to Video","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","25 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 25 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-arena-web-coding","name":"SuperCLUE Arena Web Coding","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (40 entries)."],["scores.models","pass","32 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3.1 Pro (#17)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 33 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-200k","name":"SuperCLUE-200K","summary":{"pass":6,"warn":2,"fail":0,"info":4,"text":"6 of 8 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 1 model we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-4 Turbo (Jan 2024)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-adsvu","name":"SuperCLUE-AdsVU","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-agent","name":"SuperCLUE-Agent","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","7 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for Qwen2-72B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 7 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-agent-scenario","name":"SuperCLUE-Agent Scenario","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek V4.1 Flash 1017 (#23)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-aisearch","name":"SuperCLUE-AISearch","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 14 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-asr","name":"SuperCLUE-ASR","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","3 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.5 Transcribe."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on superclueai.com: 3 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-auto","name":"SuperCLUE-Auto","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","3 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in October 2023, for ChatGLM3-6B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen-14B (#214)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-code-scenario","name":"SuperCLUE-Code Scenario","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek V4.1 Flash 1017 (#23)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-code3","name":"SuperCLUE-Code3","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","8 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-3.5 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","pass","The best is 68%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-coder","name":"SuperCLUE-Coder","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-comicshorts","name":"SuperCLUE-ComicShorts","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 1 row for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-cot","name":"SuperCLUE-CoT","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries)."],["scores.models","pass","14 models we track, plus 22 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for DeepSeek-R1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 14 rows for models we track and 22 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-cpif","name":"SuperCLUE-CPIF","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","12 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 12 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-cpifopen","name":"SuperCLUE-CPIFOpen","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-deepresearch","name":"SuperCLUE-DeepResearch","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-deepsearch","name":"SuperCLUE-DeepSearch","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","8 models we track, plus 16 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 8 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-deepswe","name":"SuperCLUE-DeepSWE","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","2 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-dspsafebench","name":"SuperCLUE-DSPSafeBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","4 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for MiniCPM3-4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is phi-3-mini 3.8B (#204)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 4 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-edit","name":"SuperCLUE-Edit","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","7 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 7 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-fact","name":"SuperCLUE-Fact","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","7 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 7 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-faith","name":"SuperCLUE-Faith","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","8 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for MiniMax-M2.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 8 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-fin","name":"SuperCLUE-Fin","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-hard","name":"SuperCLUE-Hard","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (40 entries)."],["scores.models","pass","13 models we track, plus 27 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for DeepSeek-V3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on cluebenchmarks.com: 13 rows for models we track and 27 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-i2v","name":"SuperCLUE-I2V","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-icabin","name":"SuperCLUE-ICabin","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","3 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#167)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 3 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-image","name":"SuperCLUE-Image","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","10 models we track, plus 15 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 10 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-industry","name":"SuperCLUE-Industry","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","6 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Qwen1.5-14B-Chat."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 6 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-larkclaw","name":"SuperCLUE-LarkClaw","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 5 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-law","name":"SuperCLUE-Law","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-long","name":"SuperCLUE-Long","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","1 model we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 1 row for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-longcontext","name":"SuperCLUE-LongContext","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","13 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Qwen4-Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3.1 Pro (#17)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 13 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-math6","name":"SuperCLUE-Math6","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (11 entries) and system cards (1 score)."],["scores.models","pass","5 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for DeepSeek-V2 (MoE-236B, May 2024)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on cluebenchmarks.com: 4 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-math6o","name":"SuperCLUE-Math6o","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries)."],["scores.models","pass","14 models we track, plus 23 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for DeepSeek-R1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 14 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-medassist","name":"SuperCLUE-MedAssist","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-mkt","name":"SuperCLUE-Mkt","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for DeepSeek-V3.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3.2 (#91)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 8 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-office","name":"SuperCLUE-Office","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 3 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-ondevice","name":"SuperCLUE-OnDevice","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-open","name":"SuperCLUE-Open","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","3 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-3.5 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-openclaw","name":"SuperCLUE-OpenClaw","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-project","name":"SuperCLUE-Project","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","16 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-235B-A22B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 16 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-r1-search","name":"SuperCLUE-R1 Search","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-r2v","name":"SuperCLUE-R2V","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","2 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 2 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-rag","name":"SuperCLUE-RAG","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","1 model we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 1 row for models we track and 9 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-role","name":"SuperCLUE-Role","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","7 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Qwen1.5-14B-Chat."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 7 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-safety","name":"SuperCLUE-Safety","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","7 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for Yi-34B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 7 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-science","name":"SuperCLUE-Science","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","16 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-235B-A22B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 16 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-swe","name":"SuperCLUE-SWE","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","16 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for GLM-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 16 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-swe-with-claude-code","name":"SuperCLUE-SWE with Claude Code","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-swe-with-mini-swe-agent","name":"SuperCLUE-SWE with mini-swe-agent","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","8 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for LongCat-2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek V4.1 Flash 1017 (#23)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 8 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-terminal","name":"SuperCLUE-Terminal","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 models we track."],["scores.recent","pass","The newest is from September 2026, for MiMo-V2.6 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-tts","name":"SuperCLUE-TTS","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","4 models we track, plus 13 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 4 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-v","name":"SuperCLUE-V","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","6 models we track, plus 11 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Gemini 1.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 6 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-video","name":"SuperCLUE-Video","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","6 models we track, plus 26 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Wan2.1-T2V-14B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 6 rows for models we track and 26 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-videoedit","name":"SuperCLUE-VideoEdit","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","3 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 3 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-vlm","name":"SuperCLUE-VLM","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Tencent Hy4."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-vlr","name":"SuperCLUE-VLR","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","4 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 4 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-voice-audiovisual-interaction","name":"SuperCLUE-Voice (audiovisual interaction)","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-voice-speech-interaction","name":"SuperCLUE-Voice (speech interaction)","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","1 model we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in November 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#161)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 1 row for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-webcoding","name":"SuperCLUE-WebCoding","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for MiniMax-M2.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-weclaw","name":"SuperCLUE-WeClaw","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 5 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-world","name":"SuperCLUE-World","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 4 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-writing","name":"SuperCLUE-Writing","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","12 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for ERNIE 5.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 12 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superclue-xclaw","name":"SuperCLUE-XClaw","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on superclueai.com: 6 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superglue","name":"SuperGLUE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries) and claims Epoch AI collected (8 models)."],["scores.models","pass","10 models we track, plus 23 other entries on its leaderboard."],["scores.recent","info","The newest came 4 years ago, in April 2022, for PaLM (540B). That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 90.4%, past 85% since October 2019."],["import.importer","pass","We read the board on super.gluebenchmark.com: 3 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"supergpqa","name":"SuperGPQA","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (90 entries), launch posts (30 scores) and system cards (14 scores)."],["scores.models","pass","94 models we track, plus 31 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 Pro (#32)."],["scores.headroom","pass","The best is 73.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on supergpqa.github.io: 70 rows for models we track and 31 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superlim","name":"SuperLim 1.0","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"superlim-2","name":"SuperLim 2.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on lab.kb.se: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"supermatbench","name":"SuperMatBench","summary":{"pass":4,"warn":0,"fail":1,"info":5,"text":"4 of 5 checks pass, 1 problem"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","fail","Unverified. We haven’t checked it against a primary source yet."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Ling-3.1-flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 51.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"supersim","name":"SuperSim","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"surds","name":"SURDS","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries) and launch posts (2 scores)."],["scores.models","pass","9 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 77.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"surface-evolver-bench","name":"Surface Evolver Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","25 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 95%, past 85% since April 2026."],["import.importer","pass","We read the board on yhenon.github.io: 27 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"surge-ai-blind-coding-evaluation","name":"Surge AI blind coding evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"subesco","name":"SUST Bangla Emotional Speech Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"susvibes","name":"SusVibes","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","11 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 43.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on leililab.github.io: 24 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sv-gap-reset-release","name":"SV-Gap Reset Release","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"svamp","name":"SVAMP","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","An importer exists, but it hasn’t read the board yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sveaknow-culture","name":"SveaKnow Culture","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","13 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Granite 4.2 30B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 27B (#59)."],["scores.headroom","info","Saturated. The best is 95.3%, past 85% since December 2024."],["import.importer","pass","We read the board on markovic.com: 13 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"svg-bench-minimax","name":"SVG-Bench (MiniMax)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is MiniMax-M3 (#80)."],["scores.headroom","pass","The best is 63.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"svg-bench-starvector","name":"SVG-Bench (StarVector)","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on starvector.github.io: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swag","name":"SWAG","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (24 entries)."],["scores.models","pass","24 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 24 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swarm-bench-moonshot","name":"Swarm Bench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 76.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-agentic-fabrication-eval-google","name":"SWE agentic fabrication eval","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-atlas","name":"SWE Atlas","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries), launch posts (14 scores) and system cards (2 scores)."],["scores.models","pass","36 models we track."],["scores.recent","pass","The newest is from November 2026, for SWE-2 Flash."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 84%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 24 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-atlas-codebase-qna-extended-mercor","name":"SWE Atlas Codebase QnA Extended","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (38 entries)."],["scores.models","pass","38 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 44.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mercor.com: 38 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-atlas-refactoring","name":"SWE Atlas Refactoring","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries) and launch posts (1 score)."],["scores.models","pass","18 models we track."],["scores.recent","pass","The newest is from November 2026, for LongCat-2.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 59%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 17 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-atlas-test-writing","name":"SWE Atlas Test Writing","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries) and launch posts (1 score)."],["scores.models","pass","23 models we track."],["scores.recent","pass","The newest is from November 2026, for LongCat-2.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 67%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 24 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-atlas-1-1","name":"SWE Atlas v1.1","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","31 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 80.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 31 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-refactor-bench","name":"SWE Refactor Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries) and launch posts (1 score)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 47%, under 85%, so it still separates models."],["import.importer","pass","We read the board on lab.einsia.ai: 34 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench","name":"SWE-bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on swebench.com: 15 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","1 finding, all fixed."]]},{"id":"swe-bench-cli-snorkel","name":"SWE-bench CLI","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 16.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snorkel.ai: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-lite","name":"SWE-bench Lite","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (63 entries)."],["scores.models","pass","63 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on swebench.com: 56 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-multilingual","name":"SWE-bench Multilingual","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries), launch posts (67 scores) and system cards (39 scores)."],["scores.models","pass","81 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 96.6%, past 85% since April 2026."],["import.importer","pass","We read the board on swebench.com: 13 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 6 pairs differs by more than 10 points, the largest for DeepSeek-V3.2 (+11 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-multimodal","name":"SWE-bench Multimodal","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries), launch posts (11 scores) and system cards (10 scores)."],["scores.models","pass","14 models we track, plus 19 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 74.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on swebench.com: 20 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-multimodal-v2","name":"SWE-bench Multimodal v2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries) and system cards (1 score)."],["scores.models","pass","23 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","7 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 70.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 22 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-pro","name":"SWE-Bench Pro","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries), launch posts (77 scores) and system cards (37 scores)."],["scores.models","pass","98 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 93%, past 85% since September 2026."],["import.importer","pass","We read the board on labs.scale.com: 24 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","2 large gaps of 7 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-pro-private","name":"SWE-Bench Pro Private","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from July 2026, for Muse Spark 1.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 51.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 14 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-pro-v2","name":"SWE-Bench Pro V2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries) and launch posts (1 score)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Small."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 99.4%, past 85% since June 2026."],["import.importer","pass","We read the board on labs.scale.com: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-pro-v2-hard","name":"SWE-Bench Pro V2 Hard","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries), launch posts (4 scores) and system cards (1 score)."],["scores.models","pass","21 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 98%, past 85% since June 2026."],["import.importer","pass","We read the board on labs.scale.com: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-pro-verified-opencompass","name":"SWE-Bench Pro Verified","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-promax","name":"SWE-Bench ProMax","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 41.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-science","name":"SWE-bench Science","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","11 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 50.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on swescience.github.io: 12 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-verified","name":"SWE-bench Verified","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (138 entries), Epoch AI’s runs (32 models), launch posts (144 scores) and system cards (80 scores)."],["scores.models","pass","176 models we track, plus 94 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 96%, past 85% since April 2026."],["import.importer","pass","We read the board on swebench.com: 135 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 42 pairs differs by more than 10 points, the largest for Devstral Small 2 (+12 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-verified-hard-subset-anthropic","name":"SWE-bench Verified (hard subset)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (9 scores)."],["scores.models","pass","9 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 48.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-verified-extended-mercor","name":"SWE-bench Verified Extended","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries)."],["scores.models","pass","44 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 83.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mercor.com: 45 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-verified-hard-meta","name":"SWE-bench Verified hard subset (Meta)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for Muse Spark."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark (#54)."],["scores.headroom","pass","The best is 42.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-live","name":"SWE-bench-Live","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries) and launch posts (1 score)."],["scores.models","pass","12 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 71%, under 85%, so it still separates models."],["import.importer","pass","We read the board on swe-bench-live.github.io: 23 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-bench-live-verified","name":"SWE-bench-Live Verified","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 40%, under 85%, so it still separates models."],["import.importer","pass","We read the board on swe-bench-live.github.io: 2 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-ci","name":"SWE-CI","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","16 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Qwen3.5-Plus. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Saturated. The best is 92%, past 85% since February 2026."],["import.importer","pass","We read the board on github.com: 16 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-clar","name":"SWE-Clar","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 67%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 9 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-cowork","name":"SWE-CoWork","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 7.92%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tokenwave-ai.github.io: 10 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-dev","name":"SWE-Dev","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Kimi K2. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 66.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-evo","name":"SWE-EVO","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","19 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 39.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 56 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-explore","name":"SWE-Explore","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Kimi K2.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 21.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-fficiency","name":"SWE-fficiency","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries) and launch posts (1 score)."],["scores.models","pass","18 models we track."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 34.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on swefficiency.com: 17 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-game","name":"SWE-Game","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-gen-js","name":"SWE-gen JS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-interact","name":"SWE-Interact","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 50.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-lancer","name":"SWE-Lancer","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries), launch posts (6 scores) and system cards (9 scores)."],["scores.models","pass","15 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Seed2.0 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 81.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-lancer-manager","name":"SWE-Lancer Diamond (SWE Manager)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries) and system cards (4 scores)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for o3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 51%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-marathon","name":"SWE-Marathon","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries), launch posts (9 scores) and system cards (6 scores)."],["scores.models","pass","21 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 72.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on swe-marathon.org: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-marathon-1-1","name":"SWE-Marathon v1.1","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries) and launch posts (1 score)."],["scores.models","pass","16 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","7 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 53.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on swe-marathon.org: 20 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-marathon-ext-mercor","name":"SWE-Marathon-Ext","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 74%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mercor.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-pair","name":"SWE-Pair","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Saturated. The best is 90.4%, past 85% since September 2026."],["import.importer","pass","We read the board on labs.evaligo.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-perf","name":"SWE-Perf","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries) and launch posts (1 score)."],["scores.models","pass","7 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for MiniMax-M2.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 3.10%, under 85%, so it still separates models."],["import.importer","pass","We read the board on swe-perf.github.io: 7 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-pi-meta","name":"SWE-PI","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-polybench","name":"SWE-PolyBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for DeepSeek-R1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-R1 (#130)."],["scores.headroom","pass","The best is 14.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on amazon-science.github.io: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-race","name":"SWE-Race","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Luna (#36)."],["scores.headroom","pass","The best is 81.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.evaligo.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-rebench","name":"SWE-rebench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries) and system cards (1 score)."],["scores.models","pass","14 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 64.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on swe-rebench.com: 13 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-rebench-v2","name":"SWE-rebench V2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for MiniMax-M2.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 25.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-review","name":"SWE-Review","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and launch posts (1 score)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for MiniMax-M2.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","pass","The best is 16.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-serve","name":"SWE-Serve","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 75.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on research.nvidia.com: 30 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-smith","name":"SWE-smith","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Claude Sonnet 4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 4 (#113)."],["scores.headroom","pass","The best is 38%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-sweep","name":"SWE-sweep","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries), launch posts (3 scores) and system cards (2 scores)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","4 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 13.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on swesweep.com: 18 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swe-together","name":"SWE-Together","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","22 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 69.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on togetherbench.com: 22 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swebbench-iflytek","name":"SWebBench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Spark-X2.6-4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 71.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swediagnostics","name":"SweDiagnostics","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on lab.kb.se: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swedish-analogy","name":"Swedish Analogy Test Set","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on lab.kb.se: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swedish-benchmarks-markovic","name":"Swedish Benchmarks V1.0","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","12 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Qwen3.6 27B. That’s expected for a test replaced by Swedish Benchmarks V1.1."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.6 27B (#88)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on markovic.com: 12 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swedish-benchmarks-markovic-1-1","name":"Swedish Benchmarks V1.1","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","13 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Granite 4.2 30B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 27B (#59)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on markovic.com: 13 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swefaq-2","name":"SweFAQ 2.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on lab.kb.se: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swenli","name":"SweNLI","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on lab.kb.se: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sweparaphrase","name":"SweParaphrase 1.0","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sweparaphrase-2","name":"SweParaphrase 2.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on lab.kb.se: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swesat-synonyms","name":"SweSAT Synonyms 1.0","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swesat-synonyms-1-1","name":"SweSAT Synonyms 1.1","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on lab.kb.se: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swewic","name":"SweWiC","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on lab.kb.se: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swewinogender-2","name":"SweWinogender 2.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on lab.kb.se: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swewinograd-2","name":"SweWinograd 2.0","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on lab.kb.se: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"gnehm-ict-term-recognition","name":"Swiss Job Market Monitor ICT Term Recognition","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"smartvote-2023-model-party-agreement","name":"Swiss Smartvote 2023 party agreement","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (65 entries)."],["scores.models","pass","51 models we track, plus 14 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Qwen 3.6 Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 80.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 52 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swiss-judgment-prediction","name":"Swiss-Judgment-Prediction","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"switzerlandqa-swiss-ai","name":"SwitzerlandQA","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (36 entries) and system cards (1 score)."],["scores.models","pass","9 models we track, plus 28 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Apertus 70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","pass","The best is 74.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 28 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swivuriso","name":"Swivuriso","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in October 2024, for Whisper large-v3-turbo."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swm-bench-social-world-models","name":"SWM-Bench (Social World Models)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"swt-bench","name":"SWT-bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries) and launch posts (1 score)."],["scores.models","pass","9 models we track, plus 5 other entries on its leaderboard."],["scores.recent","info","The newest came 11 months ago, in December 2025, for MiniMax-M2.1. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 mini (#92)."],["scores.headroom","info","Saturated. The best is 89.1%, past 85% since December 2025."],["import.importer","pass","We read the board on swtbench.com: 9 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sycobench-600","name":"SycoBench-600","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.5 Flash (#112)."],["scores.headroom","info","Saturated. The best is 95.3%, past 85% since May 2025."],["import.importer","pass","We read the board on github.com: 7 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sycophancy-offline-eval-openai","name":"Sycophancy evaluation (offline)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sycophancy-on-user-provided-prompts-anthropic","name":"Sycophancy on user-provided prompts (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Claude Opus 4.5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","pass","The best is 10%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sycophancy-prompts-anthropic","name":"Sycophancy prompts (Anthropic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sycophancy-eval-answer","name":"SycophancyEval (answer sycophancy)","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Grok 4.20."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4.20 (#61)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sycophancyeval","name":"SycophancyEval (Sharma et al.)","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Claude Opus 4."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sydney-captions","name":"Sydney-Captions","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"symptemist","name":"SympTEMIST","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"synergy-systematic-review-screening","name":"SYNERGY systematic review screening","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since February 2026."],["import.importer","pass","We read the board on 67AI Lab: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"synergy-plus","name":"SYNERGY+","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"synlogic","name":"SynLogic","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"synthbench-survey","name":"SynthBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","16 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Sonnet 5.5 (#9)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on synthbench.org: 43 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"synthdocbench","name":"SynthDocBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 72.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sarvam-synthetic-contextual-asr","name":"Synthetic Contextual ASR Benchmark (Indic)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ssmr-bench","name":"Synthetic Sheet Music Reasoning Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","8 models we track, plus 5 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 90.5%, past 85% since February 2026."],["import.importer","pass","We read the board on MuTheoryEval: 10 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"sysbench","name":"SysBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries), launch posts (4 scores) and system cards (5 scores)."],["scores.models","pass","13 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 81.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"szeged-ner","name":"Szeged NER","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"t-eval","name":"T-Eval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries) and launch posts (2 scores)."],["scores.models","pass","14 models we track, plus 9 other entries on its leaderboard."],["scores.recent","info","The newest came 20 months ago, in March 2025, for Hunyuan-T1. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","info","Saturated. The best is 86.4%, past 85% since November 2023."],["import.importer","pass","We read the board on open-compass.github.io: 12 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"t2i-nsi","name":"T2I-NSI","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"t2lsc-bench","name":"T2LSC-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Gemini 3.1 Flash Image. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 99%, past 85% since February 2026."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"t2ranking","name":"T2Ranking","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"taarofbench","name":"TAAROFBENCH","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 1 model we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for DeepSeek-V3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3 (#141)."],["scores.headroom","pass","The best is 56.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tabfact","name":"TabFact","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"table-judge-benchmark","name":"Table Judge Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Saturated. The best is 94.9%, past 85% since February 2026."],["import.importer","pass","We read the board on github.com: 8 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tablebench","name":"TableBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (49 entries)."],["scores.models","pass","12 models we track, plus 37 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 61.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tablebench.github.io: 29 rows for models we track and 37 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tac-2017-adr","name":"TAC 2017 Adverse Drug Reaction Extraction from Drug Labels","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tacit-knowledge-troubleshooting-openai","name":"Tacit Knowledge and Troubleshooting","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (34 scores)."],["scores.models","pass","30 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-Rosalind-6."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 94%, past 85% since September 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tacit-knowledge-brainstorm-openai","name":"Tacit Knowledge Brainstorm (Open-Ended)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 22 months ago, in January 2025, for o3-mini. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 51%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tacmind-bench","name":"TacMind-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on tacmind-bench.github.io: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tacos","name":"TACoS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Seed1.5-VL. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 49.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tailor-bench","name":"TailOR","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tmsv-speech-enhancement","name":"Taiwan Mandarin Speech with Video: Speech Enhancement","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"taiwan-dental-technician-exam-2023","name":"Taiwan National Dental Technician Licensing Examination 2023","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on pmc.ncbi.nlm.nih.gov: 3 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tmlu","name":"Taiwanese Mandarin Language Understanding","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (24 entries)."],["scores.models","pass","11 models we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for GPT-4 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#172)."],["scores.headroom","pass","The best is 73.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 11 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"takeoffbench-v1","name":"TakeoffBench-v1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","6 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for GLM-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 61.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on handoff.ai: 6 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"talentclef-2025-job-skill-prediction","name":"TalentCLEF 2025 Job Title-Based Skill Prediction","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on talentclef.github.io: 1 row for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"talentclef-2025-job-title-matching","name":"TalentCLEF 2025 Multilingual Job Title Matching","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on talentclef.github.io: 16 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"talentclef-2026-job-person-matching","name":"TalentCLEF 2026 Contextualized Job-Person Matching","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on talentclef.github.io: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"talentclef-2026-job-skill-matching","name":"TalentCLEF 2026 Job-Skill Matching with Skill Type Classification","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on talentclef.github.io: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"talk-to-act","name":"Talk to Act","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","2 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Gemma 4 E2B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 63.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"isd-talking-points-russian-state-media","name":"Talking Points: When Chatbots Surface Russian State Media","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","warn","The record links a board on isdglobal.org that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tallyqa","name":"TallyQA","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Amazon Nova Premier. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","pass","The best is 61.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tamperbench-far-ai","name":"TamperBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tap-vid","name":"TAP-Vid","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on tapvid.github.io: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tarantubench","name":"TarantuBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tartanair-v2","name":"TartanAir v2","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"task-aw-it-support","name":"Task AW IT support","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"output-homogenization-eval","name":"Task-dependent output homogenization evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for Muse Spark."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark (#54)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"taskbench","name":"TaskBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","4 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GPT-3.5 Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 2 (#192)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"taste","name":"TASTE","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 60%, under 85%, so it still separates models."],["import.importer","pass","We read the board on www-cdn.anthropic.com: 19 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"taste-bench","name":"Taste-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 59.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 14 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tasteval","name":"TasteVal","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","pass","We read the board on pzeroresearch.com: 20 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tat-dqa","name":"TAT-DQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for Gemini 1.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#174)."],["scores.headroom","pass","The best is 37.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on nextplusplus.github.io: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tat-qa","name":"TAT-QA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","27 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on nextplusplus.github.io: 27 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tatoeba-sentence-retrieval","name":"Tatoeba sentence retrieval","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (187 entries)."],["scores.models","pass","5 models we track, plus 182 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for EmbeddingGemma 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 99.5%, past 85% since March 2026."],["import.importer","pass","We read the board on MTEB: 5 rows for models we track and 182 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tatoeba-translation-challenge","name":"Tatoeba Translation Challenge","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau-urban-acoustic-scenes-2022-mobile","name":"TAU Urban Acoustic Scenes 2022 Mobile","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau-bench","name":"tau-bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries), launch posts (20 scores) and system cards (6 scores)."],["scores.models","pass","28 models we track, plus 7 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Qwen3-Next-80B-A3B. That’s expected for a test replaced by tau2-bench."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 70.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau-bench-airline","name":"tau-bench Airline","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in October 2024, for Claude 3.5 Sonnet (October 2024)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (October 2024) (#150)."],["scores.headroom","pass","The best is 46%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau-bench-retail","name":"tau-bench Retail","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries) and launch posts (2 scores)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for gpt-oss-120b."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is gpt-oss-120b (#126)."],["scores.headroom","pass","The best is 69.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau2-airline-code-agents-snorkel","name":"Tau2 Bench Airline With Code Agents","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau2-verified-airline-code-agents-snorkel","name":"Tau2 Bench Verified Airline With Code Agents","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau2-bench","name":"tau2-bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries), launch posts (81 scores) and system cards (20 scores)."],["scores.models","pass","85 models we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 98.9%, past 85% since October 2025."],["import.importer","pass","We read the board on taubench.com: 11 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 6 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau2-bench-verified","name":"tau2-bench Verified","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries) and launch posts (3 scores)."],["scores.models","pass","12 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 82.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 14 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau2-noise-longcat","name":"tau2-Noise (LongCat)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 10 months ago, in January 2026, for LongCat-Flash-Thinking-2601. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 67.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau3-bench","name":"tau3-bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries), launch posts (14 scores) and system cards (1 score)."],["scores.models","pass","21 models we track."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Saturated. The best is 96%, past 85% since November 2025."],["import.importer","pass","We read the board on taubench.com: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tavernbench","name":"TavernBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tavily-company-search-benchmark","name":"Tavily Company Search Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on tavily.com: 9 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-tax-agent-bench","name":"Tax Agent Bench","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (88 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","88 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 52.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 88 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"taxabench-8k","name":"TaxaBench-8k","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"taxcalcbench","name":"TaxCalcBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4. That’s expected for a test replaced by TaxCalcBench v2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 Pro (#19)."],["scores.headroom","pass","The best is 62.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on taxcalcbench.ai: 19 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"taxcalcbench-2","name":"TaxCalcBench v2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","38 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 86%, past 85% since November 2026."],["import.importer","pass","We read the board on taxcalcbench.ai: 70 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-taxeval-v2","name":"TaxEval v2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (131 entries)."],["scores.models","pass","129 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","pass","The best is 80.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 143 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"taxobench","name":"TaxoBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tcs-bench","name":"TCS-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","3 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 55%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tcsexam","name":"TCSExam","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on moonshotai.github.io: 7 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tcsexam-lite","name":"TCSExam-Lite","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"te-hiku-maori-golden-dataset","name":"Te Hiku Media Māori speech recognition golden dataset","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on blog.papareo.nz: 3 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"te-ca","name":"TE-ca","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"teach","name":"TEACh","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"teacharena","name":"TeachArena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries)."],["scores.models","pass","16 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for GLM-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 Pro (#15)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 16 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"teaching-monster-challenge","name":"Teaching Monster Challenge","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on teaching.monster: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"teachobs","name":"TeachObs","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Grok 4.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ted-lium","name":"TED-LIUM","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"telbench","name":"TELBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 54.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on nju-link.github.io: 3 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"teleqna","name":"TeleQnA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (90 entries)."],["scores.models","pass","69 models we track, plus 21 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Saturated. The best is 85.2%, past 85% since February 2026."],["import.importer","pass","We read the board on huggingface.co: 70 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"teletables","name":"TeleTables","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (90 entries)."],["scores.models","pass","69 models we track, plus 21 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 48%, under 85%, so it still separates models."],["import.importer","pass","We read the board on GSMA Open Telco Leaderboard: 70 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"telugumixbench","name":"TeluguMixBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Haiku 5.5 (#20)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since October 2025."],["import.importer","pass","We read the board on kaggle.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tem-vqa","name":"TEM-VQA","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","8 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#112)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tempcompass","name":"TempCompass","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries), launch posts (7 scores) and system cards (5 scores)."],["scores.models","pass","20 models we track, plus 17 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Seed1.8. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 89.6%, past 85% since December 2025."],["import.importer","pass","We read the board on huggingface.co: 13 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tool-template-following-qwen","name":"Template Following generalization (Qwen)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Qwen3-Coder-Next. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 92.7%, past 85% since February 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"temporal-reasoning-search-eval-google","name":"Temporal reasoning with search","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"temporalbench","name":"TemporalBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","11 models we track, plus 21 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Seed1.5-VL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 79.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on temporalbench.github.io: 11 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"workbuddy-bench-tencent","name":"Tencent WorkBuddy Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 81.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on workbuddybench.com: 19 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tennis-padel-state-tracking-benchmark","name":"Tennis & Padel State Tracking Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 4 26B A4B (#115)."],["scores.headroom","pass","The best is 0%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terminal-bench","name":"Terminal-Bench","summary":{"pass":6,"warn":1,"fail":0,"info":3,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from boards Epoch AI copied (18 models), launch posts (20 scores) and system cards (8 scores)."],["scores.models","pass","33 models we track."],["scores.recent","info","The newest came 11 months ago, in December 2025, for Tencent HY 2.0 Think. That’s expected for a test replaced by Terminal-Bench 2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 50%, under 85%, so it still separates models."],["import.importer","warn","The record links a board on tbench.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terminal-bench-2-0","name":"Terminal-Bench 2.0","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (113 entries), launch posts (62 scores), system cards (28 scores) and claims Epoch AI collected (5 models)."],["scores.models","pass","84 models we track, plus 78 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Laguna XS 2.1."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 82.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tbench.ai: 139 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 14 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terminal-bench-2-0-verified-zai","name":"Terminal-Bench 2.0 Verified (Z.ai)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for GLM-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","pass","The best is 61.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 8 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terminal-bench-2-1","name":"Terminal-Bench 2.1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries), launch posts (59 scores) and system cards (23 scores)."],["scores.models","pass","64 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 92.8%, past 85% since June 2026."],["import.importer","pass","We read the board on tbench.ai: 22 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 6 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terminal-bench-2-1-extended-mercor","name":"Terminal-Bench 2.1 Extended","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (46 entries)."],["scores.models","pass","44 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 54%, under 85%, so it still separates models."],["import.importer","pass","We read the board on mercor.com: 47 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terminal-bench-3-0","name":"Terminal-Bench 3.0","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries), launch posts (6 scores) and system cards (4 scores)."],["scores.models","pass","12 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 43.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tbench.ai: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terminal-bench-4-0","name":"Terminal-Bench 4.0","summary":{"pass":11,"warn":1,"fail":0,"info":0,"text":"11 of 12 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (44 entries), launch posts (29 scores) and system cards (15 scores)."],["scores.models","pass","52 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 70.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tbench.ai: 57 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 13 pairs differs by more than 10 points, the largest for MiMo-V2.6 Flash (+13 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terminal-bench-4-1","name":"Terminal-Bench 4.1","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (52 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","43 models we track, plus 10 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 71.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tbench.ai: 65 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terminal-bench-challenges","name":"Terminal-Bench Challenges","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terminal-bench-data-turing","name":"Terminal-Bench Data","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 48.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.turing.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terminal-bench-hard","name":"Terminal-Bench Hard","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (9 scores) and system cards (9 scores)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 4 31B IT (#101)."],["scores.headroom","pass","The best is 36%, under 85%, so it still separates models."],["import.importer","info","Artificial Analysis allows reuse of only its index numbers, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tb-mathmodel","name":"Terminal-Bench Math Modeling","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","8 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on wxj630.github.io: 10 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terminal-bench-pro","name":"Terminal-Bench Pro","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","23 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5 397B-A17B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 51%, under 85%, so it still separates models."],["import.importer","pass","We read the board on alibaba.github.io: 23 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terminal-bench-science","name":"Terminal-Bench-Science","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries), launch posts (14 scores) and system cards (9 scores)."],["scores.models","pass","27 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 77.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on terminal-bench-science.ai: 17 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 3 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terminal-x","name":"Terminal-X","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.7 (#31)."],["scores.headroom","pass","The best is 42.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on unipat.ai: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terminalworld","name":"TerminalWorld","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 62.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on terminalworld.ai: 8 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terms-bench","name":"TERMS-Bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","11 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on terms-bench.github.io: 11 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terrafirm","name":"TerraFirm REIT","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"terretaqa","name":"TerretaQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"testing-model-s-emotional-inteligence","name":"Testing model's emotional inteligence","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is Claude Sonnet 5.5 (#9)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2025."],["import.importer","pass","We read the board on kaggle.com: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"testing-relational-understanding-in-text-guided-image-generation","name":"Testing Relational Understanding in Text-Guided Image Generation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"testprism","name":"TestPrism","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Fable 5.1 (#8)."],["scores.headroom","pass","The best is 33.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 24 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tetrisbench","name":"TetrisBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 66.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tetrisbench.com: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"text-classification-bytedance","name":"Text Classification (ByteDance Seed in-house)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Seed2.0 Pro. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 69%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"political-persuasion-text-study-openai","name":"Text persuasion study on political opinions","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"text-to-text-safety-google","name":"Text to Text Safety","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"text-based-rl-task-anthropic","name":"Text-based reinforcement learning task (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (7 scores)."],["scores.models","pass","7 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"text2cad","name":"Text2CAD","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"textarena","name":"TextArena","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on textarena.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"textcaps","name":"TextCaps","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini Embedding 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"textedit-opencompass","name":"TextEdit","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"textedit-opencompass-real","name":"TextEdit Real","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","3 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"textedit-opencompass-virtual","name":"TextEdit Virtual","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","3 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"textquests","name":"TextQuests","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (36 entries) and system cards (1 score)."],["scores.models","pass","32 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on textquests.ai: 62 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"textvbench","name":"TextVBench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","6 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Cosmos3-Nano."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"textvqa","name":"TextVQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries), launch posts (11 scores) and system cards (11 scores)."],["scores.models","pass","18 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-V-4.7-35B-A3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","pass","The best is 85%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"textworld-cooking-gata","name":"TextWorld Cooking (GATA)","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for Qwen3-4B-Thinking-2507."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on Prime Intellect: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tgif-qa","name":"TGIF-QA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tgve-plus","name":"TGVE+","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"thaiexam","name":"ThaiExam","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries)."],["scores.models","pass","24 models we track, plus 18 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for Command R+."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","pass","The best is 75.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on crfm.stanford.edu: 24 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-aggregate-asteroid-numbers","name":"The Aggregate Asteroid Numbers","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 58.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on theaggregate.ai: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-aggregate-chart-recall","name":"The Aggregate Chart Recall","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on theaggregate.ai: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-aggregate-day-one-recall-check","name":"The Aggregate Day-one Recall Check","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries)."],["scores.models","pass","36 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 61.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on theaggregate.ai: 36 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-aggregate-exotic-fact-recall","name":"The Aggregate Exotic Fact Recall","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 75%, under 85%, so it still separates models."],["import.importer","pass","We read the board on theaggregate.ai: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-aggregate-forecasting-resolved-questions","name":"The Aggregate Forecasting Resolved Questions","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a Brier score, with no fixed ceiling."],["import.importer","pass","We read the board on theaggregate.ai: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-aggregate-inversion-and-execution-ladder","name":"The Aggregate Inversion and Execution Ladder","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","20 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Flint."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since December 2025."],["import.importer","pass","We read the board on theaggregate.ai: 20 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-aggregate-lichess-pairwise-strength-25-pairs","name":"The Aggregate Lichess Pairwise Strength","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 88%, past 85% since September 2026."],["import.importer","pass","We read the board on theaggregate.ai: 19 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-aggregate-lichess-rating-prediction-25-games","name":"The Aggregate Lichess Rating Prediction","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on theaggregate.ai: 19 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-aggregate-long-tail-book-recall","name":"The Aggregate Long-tail Book Recall","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","21 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 77%, under 85%, so it still separates models."],["import.importer","pass","We read the board on theaggregate.ai: 21 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-aggregate-personal-style-imitation-53-texts","name":"The Aggregate Personal Style Imitation","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 76.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on theaggregate.ai: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-aggregate-puzzle-portfolio-ladder","name":"The Aggregate Puzzle Portfolio Ladder","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","24 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 96.7%, past 85% since December 2025."],["import.importer","pass","We read the board on theaggregate.ai: 24 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-aggregate-trip-outcome-prediction","name":"The Aggregate Trip Outcome Prediction","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","26 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 69.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on theaggregate.ai: 35 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ai-teacher-test","name":"The AI Teacher Test","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-amazing-agent-race","name":"The Amazing Agent Race","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 36.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on minnesotanlp.github.io: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"genderqueer-test-suite","name":"The GenderQueer Test Suite","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ccdh-illusion-of-ai-safety","name":"The Illusion of AI Safety","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on counterharm.com: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-impossible-refund","name":"The Impossible Refund","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-met-dataset","name":"The Met Dataset","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on cmp.felk.cvut.cz: 10 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"midnight-index","name":"The Midnight Index","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on secondstrike.io that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-pushback-test","name":"The Pushback Test","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 5 (#35)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"silicon-sample-benchmark","name":"The Silicon Sample Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"the-turtle-test","name":"The Turtle Test","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on kaggle.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"theagentcompany","name":"TheAgentCompany","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 17 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for MiniMax-M2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 36%, under 85%, so it still separates models."],["import.importer","pass","We read the board on the-agent-company.com: 8 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"thegoldman-retrieval","name":"TheGoldman Retrieval","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"theone-routing-decisions","name":"TheOne Routing Decisions","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4 Nano. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 Nano (#90)."],["scores.headroom","info","Saturated. The best is 94.4%, past 85% since March 2026."],["import.importer","pass","We read the board on kaggle.com: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"theophyseval","name":"TheoPhysEval","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 98%, past 85% since September 2026."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"theoremqa","name":"TheoremQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","6 models we track, plus 30 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Qwen2.5-72B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 48.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 4 rows for models we track and 30 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"thermoqa","name":"ThermoQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for GPT-5.4. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","info","Saturated. The best is 94.1%, past 85% since May 2025."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"thinkfollow-qwen","name":"ThinkFollow (Qwen)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Qwen3-32B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","info","Saturated. The best is 98.9%, past 85% since May 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"thinkgeo","name":"ThinkGeo","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since September 2026."],["import.importer","pass","We read the board on Axis Spatial: 39 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"thinkingbox-bench","name":"Thinkingbox-bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries)."],["scores.models","pass","18 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 66.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"third-pascal-recognizing-textual-entailment-challenge","name":"Third PASCAL Recognizing Textual Entailment Challenge","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"thrive-holdings-helpdesk-ticket-resolution-benchmark","name":"Thrive Holdings helpdesk ticket resolution benchmark","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 61.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on evals.thriveholdings.com: 57 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tiif-bench","name":"TIIF-Bench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for GLM-Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"time-forecasting","name":"TIME (time series forecasting)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (48 entries)."],["scores.models","pass","48 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 48 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-time-horizon-index-ksp","name":"Time Horizon Index: KSP","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 91.3%, past 85% since September 2026."],["import.importer","pass","We read the board on vals.ai: 13 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"time-series-forecasting-task-anthropic","name":"Time series forecasting task (Anthropic)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (9 scores)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"timelens-bench","name":"TimeLens-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries) and launch posts (1 score)."],["scores.models","pass","10 models we track, plus 14 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 62.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on timelens-arc-lab.github.io: 10 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"timeline-bench","name":"TimelineBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","12 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 32.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on timelinebench.tensortest.com: 18 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"timescope-video","name":"TimeScope","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","8 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Saturated. The best is 97.3%, past 85% since June 2024."],["import.importer","pass","We read the board on huggingface.co: 8 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"timespeak","name":"TimeSpeak","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for TML-Interaction-Small. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 64.7%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tinybenchmarks","name":"tinyBenchmarks","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tir-bench","name":"TIR-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries) and launch posts (6 scores)."],["scores.models","pass","24 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen 3.6 Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.6 Plus (#75)."],["scores.headroom","pass","The best is 61.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 20 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tiresidewall-bench-multimodal-zero-shot-tire-ocr","name":"TireSidewall-Bench: Multimodal Zero-Shot Tire OCR","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 46%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tis-100-bench","name":"TIS-100 Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on tis-100.wincon.dev: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tissuemicroscopybench-latchbio","name":"TissueMicroscopyBench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","22 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 34%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 30 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"titullms-bangla-evaluation-suite","name":"TituLLMs Bangla Evaluation Suite","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tmhint-qi","name":"TMHINT-QI","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tmhint-qi-2","name":"TMHINT-QI version II","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tmmlu-plus","name":"TMMLU+","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (47 entries)."],["scores.models","pass","19 models we track, plus 28 other entries on its leaderboard."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5. That’s expected for a test replaced by TMMLU+ 1.1."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 88.6%, past 85% since August 2025."],["import.importer","pass","We read the board on huggingface.co: 19 rows for models we track and 28 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tmmlu-plus-1-1","name":"TMMLU+ 1.1","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","21 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 27B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 96.2%, past 85% since June 2026."],["import.importer","pass","We read the board on huggingface.co: 21 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toefl11-essay-proficiency","name":"TOEFL11 Essay Proficiency Scoring","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toksuite","name":"TokSuite","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"told-br","name":"ToLD-Br","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toloka-arena","name":"Toloka Arena","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 33%, under 85%, so it still separates models."],["import.importer","pass","We read the board on toloka.ai: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toloka-arena-enterprise-tool-use-2026-11","name":"Toloka Arena Enterprise Tool Use (November 2026)","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (46 entries)."],["scores.models","pass","43 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 81%, under 85%, so it still separates models."],["import.importer","pass","We read the board on toloka.ai: 43 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toloka-visual-question-answering","name":"Toloka Visual Question Answering Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on codalab.lisn.upsaclay.fr: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tomato","name":"TOMATO","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (31 entries), launch posts (4 scores) and system cards (4 scores)."],["scores.models","pass","27 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","pass","The best is 79.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 22 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tombo-full-transcript","name":"Tombo Full Transcript","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 6 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tomg-bench","name":"TOMG-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries) and launch posts (1 score)."],["scores.models","pass","6 models we track, plus 25 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Intern-S2-397B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-3.5 Turbo (#200)."],["scores.headroom","pass","The best is 66.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 25 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"refusal-tone-google","name":"Tone","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tool-use-prompt-injection-anthropic","name":"Tool use prompt injection evaluation (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 13 months ago, in October 2025, for Claude Haiku 4.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Sonnet 4.5 (#81)."],["scores.headroom","info","Saturated. The best is 96%, past 85% since September 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tool-calling-prompt-injections-openai","name":"Tool-calling prompt injections","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 15 months ago, in August 2025, for GPT-5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 99%, past 85% since August 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toolathlon","name":"Toolathlon","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (50 entries), launch posts (24 scores) and system cards (12 scores)."],["scores.models","pass","61 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 61.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on toolathlon.xyz: 51 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 18 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toolathlon-verified","name":"Toolathlon-Verified","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries), launch posts (18 scores) and system cards (9 scores)."],["scores.models","pass","40 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 80.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on toolathlon.xyz: 25 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 7 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toolbench","name":"ToolBench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toolcall-badcase-zai","name":"ToolCall-Badcase","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for GLM-5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GLM-5 (#89)."],["scores.headroom","info","Saturated. The best is 95.8%, past 85% since February 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toolcallformatifbench","name":"ToolcallFormatIFBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and launch posts (1 score)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Qwen 3.6 Max (Preview). That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.5 (#62)."],["scores.headroom","info","Saturated. The best is 86.1%, past 85% since April 2026."],["import.importer","pass","We read the board on qwen.ai: 5 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toolcomp","name":"ToolComp","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","32 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#104)."],["scores.headroom","pass","The best is 70.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 34 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toolmaze","name":"ToolMaze","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 53%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 18 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toolqa","name":"ToolQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Llama 2-13B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-70B (#210)."],["scores.headroom","pass","The best is 24.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on proceedings.neurips.cc: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toolret","name":"ToolRet","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (23 entries)."],["scores.models","pass","23 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 23 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toolsandbox","name":"ToolSandbox","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries), launch posts (5 scores) and system cards (1 score)."],["scores.models","pass","16 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 80.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 15 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tooluse-qwen","name":"ToolUse (Qwen)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for Qwen3-32B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Qwen3-32B (#135)."],["scores.headroom","info","Saturated. The best is 85.5%, past 85% since May 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"topiocqa","name":"TopiOCQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"torr","name":"ToRR","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for DeepSeek-V3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek-V3 (#141)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on crfm.stanford.edu: 14 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toxicchat","name":"ToxicChat","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toxigen","name":"ToxiGen","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (14 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Olmo 3.1 32B Think."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 2 27B (#190)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"toxigen-discriminative","name":"ToxiGen (discriminative)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Phi-4-Reasoning-plus. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 85.5%, past 85% since April 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tr-gender-bias","name":"tr_gender_bias","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (24 entries)."],["scores.models","pass","13 models we track, plus 11 other entries on its leaderboard."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Command A+. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since August 2025."],["import.importer","pass","We read the board on aclanthology.org: 13 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trace-patronus","name":"TRACE","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GLM-4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 63%, under 85%, so it still separates models."],["import.importer","pass","We read the board on proceedings.mlr.press: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tracespatial-bench","name":"TraceSpatial-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tracetrial-bench","name":"TraceTrial-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 77.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tracelite.ai: 36 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"traderank-arena","name":"TradeRank Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","19 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Tencent Hy4."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6.1 Terra (#7)."],["scores.headroom","pass","The best is 0%, under 85%, so it still separates models."],["import.importer","pass","We read the board on traderank.ai: 19 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trading-bot-bench","name":"Trading Bot Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trae-agent-bench-bytedance","name":"Trae Agent Bench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trafficimag","name":"TrafficImag","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trail-patronus","name":"TRAIL","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#123)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"training-distribution-hack-rate-anthropic","name":"Training distribution hack rate (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trait-personality","name":"TRAIT","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"traject-bench","name":"TRAJECT-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","7 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 mini (#92)."],["scores.headroom","pass","The best is 70.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trajectory-labs-cyber-red-teaming-google","name":"Trajectory Labs cyber red teaming","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"transformer-puzzles-raspy","name":"Transformer Puzzles (RASPy)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"translated-arc-hellaswag-mmlu-mistral","name":"Translated ARC-Challenge, HellaSwag and MMLU (Mistral)","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for Mixtral 8x7B."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"translation-human-scoring-zai","name":"Translation Human Scoring (Z.ai)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for GLM-5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GLM-5 (#89)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"transluce-mental-health-evaluation","name":"Transluce Mental Health Evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"transluce-user-awareness","name":"Transluce User Awareness Evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"travel-agent-compassion","name":"Travel Agent Compassion","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (36 entries)."],["scores.models","pass","34 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since December 2025."],["import.importer","pass","We read the board on compassionbench.com: 68 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"travelplanner","name":"TravelPlanner","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"treasuryqa-bench","name":"TreasuryQA Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","5 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 82.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on veris.ai: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trec-2017-liveqa-medical-task","name":"TREC 2017 LiveQA Medical Task","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on trec.nist.gov: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trec-clinical-trials-2023","name":"TREC 2023 Clinical Trials Track","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trec-biogen-2024","name":"TREC BioGen 2024","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","10 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on trec.nist.gov: 10 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trec-million-llms-2025","name":"TREC Million LLMs 2025","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on trec.nist.gov: 19 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trec-product-search-2023","name":"TREC Product Search 2023","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trec-product-search-2024","name":"TREC Product Search 2024","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on trec.nist.gov: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trec-product-search-2025","name":"TREC Product Search 2025","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","21 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on trec.nist.gov: 21 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trec-related-product-recommendation-2025","name":"TREC Related Product Recommendation 2025","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on trec.nist.gov: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trec-covid","name":"TREC-COVID","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"treebench","name":"TreeBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries), launch posts (2 scores) and system cards (7 scores)."],["scores.models","pass","17 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 71.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trek-150","name":"TREK-150","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on codalab.lisn.upsaclay.fr: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trialdesignbench","name":"TrialDesignBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"rct-clinical-evidence-extraction","name":"Trialstreamer clinical evidence extraction","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tecci","name":"Tricky Edits of Collected and Curated Images","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Flash Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 19.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on google-deepmind.github.io: 3 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"triphysbench","name":"TriPhysBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.6 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 50.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"triscore","name":"TriScore","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tritonbench","name":"TritonBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","5 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for DeepSeek-R1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 34.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trivia-et","name":"Trivia-et","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trivial-prompt-bench","name":"trivial-prompt-bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"triviaqa","name":"TriviaQA","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (19 scores), system cards (31 scores) and claims Epoch AI collected (38 models)."],["scores.models","pass","61 models we track."],["scores.recent","pass","The newest is from October 2026, for Beam 1.0 Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V4-Pro (#70)."],["scores.headroom","info","Saturated. The best is 91.8%, past 85% since February 2023."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"triworldbench","name":"TriWorldBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (69 entries)."],["scores.models","pass","69 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on triworldbench-triworldbench-space.hf.space: 69 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trlawbench","name":"TRLawBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","34 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on betaspacestudio.com: 58 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"troubleshootingbench-openai","name":"TroubleshootingBench","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (25 scores)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-Rosalind-6."],["scores.frontier","info","2 of the top 10 have a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 59%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"true-flageval","name":"TRUE (Text Recognition and Understanding)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","23 models we track."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#104)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on flageval-baai.github.io: 23 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trustboundary-bench","name":"TrustBoundary Bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#112)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"trustllm-jailbreaktrigger","name":"TrustLLM JailbreakTrigger","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries) and system cards (2 scores)."],["scores.models","pass","6 models we track, plus 12 other entries on its leaderboard."],["scores.recent","info","The newest came 11 months ago, in December 2025, for Olmo 3.1 32B Think. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 2-70B (#210)."],["scores.headroom","info","Saturated. The best is 97.4%, past 85% since July 2023."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"truthfulqa","name":"TruthfulQA","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries), launch posts (13 scores) and system cards (28 scores)."],["scores.models","pass","32 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","info","Saturated. The best is 88%, past 85% since June 2026."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"truthfulqa-multi","name":"TruthfulQA-Multi","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","6 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Llama 3.1-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-70B (#177)."],["scores.headroom","pass","The best is 50.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"truthinsightbench","name":"TruthInsightBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ts-bench","name":"ts-bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","pass","We read the board on blog.lai.so: 9 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ts-bench-v1","name":"ts-bench v1","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","2 models we track, plus 8 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for GPT-5.3 Codex. That’s expected for a test replaced by ts-bench."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Saturated. The best is 96%, past 85% since December 2025."],["import.importer","pass","We read the board on github.com: 10 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tsp-opt","name":"TSP-Opt","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries) and system cards (2 scores)."],["scores.models","pass","13 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Phi-4-Reasoning-plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 56.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 11 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tts-arena","name":"TTS Arena","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (48 entries)."],["scores.models","pass","2 models we track, plus 46 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for MiniMax Speech 2.8 HD."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 2 rows for models we track and 46 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tua-bench-meta","name":"TUA-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","12 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 65.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 39 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tuesday-work-index","name":"Tuesday Work Index","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (56 entries)."],["scores.models","pass","56 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on surgehq.ai: 57 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tulu-3-safety-eval-ai2","name":"Tulu 3 Safety Eval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (22 entries), launch posts (6 scores) and system cards (5 scores)."],["scores.models","pass","10 models we track, plus 17 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Granite 4.1 30B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o (#161)."],["scores.headroom","info","Saturated. The best is 90.9%, past 85% since September 2024."],["import.importer","pass","We read the board on huggingface.co: 5 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"turblimp","name":"TurBLiMP","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","2 models we track, plus 11 other entries on its leaderboard."],["scores.recent","info","The newest came 20 months ago, in March 2025, for Gemma 3 12B. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 93.6%, past 85% since March 2025."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"agentic-environments-turing","name":"Turing Agentic Environments","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Claude Opus 4.7."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 60.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on agentic.turing.com: 6 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"turing-tts-voicearena","name":"Turing TTS","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","4 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash-Lite TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 10.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on voicearena.com: 4 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"turkish-dental-specialty-examination-2020-2021","name":"Turkish Dental Specialty Examination 2020 and 2021","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on doi.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"turkishmmlu","name":"TurkishMMLU","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries), launch posts (1 score) and system cards (4 scores)."],["scores.models","pass","12 models we track, plus 16 other entries on its leaderboard."],["scores.recent","warn","The newest came 23 months ago, in December 2024, for Qwen2.5-14B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 79.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"turku-ner-corpus","name":"Turku NER Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"turku-paraphrase-corpus","name":"Turku Paraphrase Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"turn-based-benchmarks","name":"Turn based Benchmarks","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.7 Flash (#26)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"turnbench","name":"TurnBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","2 models we track, plus 25 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Gemini 3.1 Flash Live."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 65.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on turnbench.sesame.com: 2 rows for models we track and 25 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tut-urban-acoustic-scenes-2018","name":"TUT Urban Acoustic Scenes 2018","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tutorbench","name":"TutorBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","20 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Muse Spark."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Pro (#19)."],["scores.headroom","pass","The best is 68.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 26 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tutorcasebench","name":"TutorCaseBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 59.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tv-ad","name":"TV-AD","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tvbench","name":"TVBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (24 entries), launch posts (7 scores) and system cards (4 scores)."],["scores.models","pass","13 models we track, plus 18 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 81.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tvqa","name":"TVQA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 10 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in March 2023, for GPT-4. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Saturated. The best is 87.3%, past 85% since March 2023."],["import.importer","pass","We read the board on tvqa.cs.unc.edu: 10 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tvrbench","name":"TVRBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","7 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen3.6 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 12%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tweetsentbr","name":"tweetSentBR","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on lrec-conf.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"twitter-financial-news-sentiment","name":"Twitter Financial News Sentiment","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"twitter-url-paraphrase-corpus","name":"Twitter URL Paraphrase Corpus","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"two-hop-latent-reasoning","name":"Two-Hop Latent Reasoning","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"txbench-antibody-development","name":"TxBench-Antibody-Development","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","23 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","8 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 61.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 36 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"txbench-antibody-discovery","name":"TxBench-Antibody-Discovery","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries)."],["scores.models","pass","30 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","8 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 67.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 45 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"txbench-oligo-discovery","name":"TxBench-Oligo-Discovery","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries)."],["scores.models","pass","30 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","8 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","pass","The best is 62.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 45 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"txbench-preclinical-pharmacology","name":"TxBench-Preclinical-Pharmacology","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","32 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 68%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 48 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tydiqa","name":"TydiQA","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores) and system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 19 months ago, in April 2025, for Llama 4 Maverick. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 4 Maverick (#153)."],["scores.headroom","pass","The best is 31.7%, under 85%, so it still separates models."],["import.importer","warn","The record links a board on ai.google.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"typed-decision-bench-blobfish","name":"Typed Decision Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on blobfish.ai: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"u-artifacts-bytedance","name":"U-Artifacts","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Seed1.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 49.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"u-math","name":"U-MATH","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","30 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 22 months ago, in January 2025, for o3-mini. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1 (#118)."],["scores.headroom","info","Saturated. The best is 86.8%, past 85% since December 2024."],["import.importer","pass","We read the board on huggingface.co: 30 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"u18-safety-evals-openai","name":"U18 safety evaluations","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ua-bench","name":"UA-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","5 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Gemma 4 12B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 13.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sunzhe12.github.io: 5 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uaqfact","name":"UAQFact","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in January 2024, for GLM-4."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 49.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 1 row for models we track and 6 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uav-operator","name":"UAV Operator","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uav-robust-control-hub","name":"UAV Robust Control Hub","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ucf101","name":"UCF101","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (49 entries)."],["scores.models","pass","49 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on MTEB: 49 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uci-adult-moonshot","name":"UCI Adult Dataset (Project Moonshot)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ucm-captions","name":"UCM-Captions","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ud-ancient-greek-perseus","name":"UD Ancient Greek Perseus","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on universaldependencies.org: 26 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ud-ancient-greek-proiel","name":"UD Ancient Greek PROIEL","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on universaldependencies.org: 26 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ud-ancient-greek-ptnk","name":"UD Ancient Greek PTNK","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ud-ancient-hebrew-ptnk","name":"UD Ancient Hebrew PTNK","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ud-latin-circse","name":"UD Latin CIRCSE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 11 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ud-latin-ittb","name":"UD Latin ITTB","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on universaldependencies.org: 26 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ud-latin-llct","name":"UD Latin LLCT","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ud-latin-perseus","name":"UD Latin Perseus","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on universaldependencies.org: 26 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ud-latin-proiel","name":"UD Latin PROIEL","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on universaldependencies.org: 26 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ud-latin-udante","name":"UD Latin UDante","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ufo-bench","name":"UFO-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uganda-cultural-and-cognitive-benchmark","name":"Uganda Cultural and Cognitive Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ugmathbench","name":"UGMathBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries) and system cards (1 score)."],["scores.models","pass","8 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Ring-1T."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","pass","The best is 81.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on agi-eval.cn: 7 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uhgeval","name":"UHGEval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","6 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for GLM-4-9B-Chat."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 6 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uhura-arc-easy","name":"Uhura ARC-Easy","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for GPT-4o."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uhura-truthfulqa","name":"Uhura TruthfulQA","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for GPT-4o."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uhura-eval","name":"Uhura-Eval","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for GPT-4o."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ui-bench","name":"UI-Bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on uibench.ai: 10 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ui-genie-rm-evaluation","name":"UI-Genie reward model evaluation","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","2 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Qwen2.5-VL-72B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 66.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ui-tars-15-games-bytedance","name":"UI-TARS 15 Games Collection","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for UI-TARS-2. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ui-vision","name":"UI-Vision","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ui-vision-action-prediction","name":"UI-Vision Action Prediction","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ui-vision-element-grounding","name":"UI-Vision Element Grounding","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","5 models we track, plus 20 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 8.27%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ui-vision-layout-grounding","name":"UI-Vision Layout Grounding","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","2 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for Claude 3.7 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 24.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uit-victsd","name":"UIT-ViCTSD","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uit-vihsd","name":"UIT-ViHSD","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uit-vsfc","name":"UIT-VSFC","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uk-aisi-alignment-and-monitorability-evaluations","name":"UK AISI alignment and monitorability evaluations","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Mythos Preview. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uk-aisi-caisi-enterprise-network-range","name":"UK AISI and CAISI simulated enterprise network range","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries) and system cards (1 score)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aisi.gov.uk: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uk-aisi-correlated-monitor-failure","name":"UK AISI correlated monitor failure","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uk-aisi-cot-controllability","name":"UK AISI CoT Controllability","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uk-aisi-cyber-ranges","name":"UK AISI cyber ranges","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uk-aisi-file-download-monitor-evasion","name":"UK AISI File Download monitor evasion","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uk-aisi-narrow-cyber-tasks","name":"UK AISI narrow cyber tasks","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Saturated. The best is 95%, past 85% since April 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uk-aisi-no-cot-maths","name":"UK AISI no-CoT maths competition problems","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 79.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uk-aisi-out-of-scope-supply-chain-attack","name":"UK AISI Out of Scope Supply Chain Attack","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uk-aisi-sabotage-continuation","name":"UK AISI sabotage continuation evaluation","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uk-aisi-sandbox-escape-monitor-evasion","name":"UK AISI simulated sandbox escape monitor evasion","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uk-aisi-unprompted-sabotage","name":"UK AISI unprompted sabotage evaluation","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uk-aisi-vulnerability-discovery","name":"UK AISI vulnerability discovery challenges","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 88.6%, past 85% since November 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uk-dermatology-sce-sample-questions","name":"UK Dermatology SCE Sample Questions","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ulsa-synthesis-action-retrieval","name":"ULSA synthesis action retrieval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ultratext-bench","name":"UltraText Bench","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (24 entries)."],["scores.models","pass","17 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Flash Image."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 17 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"umi-bench","name":"UMI-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for π0.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 55.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"unauthorized-code-execution-bench-meta","name":"Unauthorized Code Execution Bench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Muse Spark 1.3 (#27)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"unauthorized-voice-generation-openai","name":"Unauthorized voice generation","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since August 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uncheatable-eval","name":"Uncheatable Eval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (95 entries)."],["scores.models","pass","33 models we track, plus 62 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 4 31B IT (#101)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 33 rows for models we track and 62 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uncritically-reporting-flawed-results-anthropic","name":"Uncritically reporting flawed results (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"unexpected-container-states-openai","name":"Unexpected container or code states","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 18 months ago, in May 2025, for codex-1. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 85%, past 85% since May 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ungrounded-inference-sensitive-trait-attribution-openai","name":"Ungrounded inference and sensitive trait attribution","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 84%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ungrounded-inference-openai","name":"Ungrounded Inference evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 19 months ago, in April 2025, for o3. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"unhealthy-comments-corpus","name":"Unhealthy Comments Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uniclawbench","name":"UniClawBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","pass","The best is 69.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on uniclawbench.github.io: 10 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"unicode-competitive-coding","name":"UniCode","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","16 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 70.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 18 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"unicorn-vision-safety","name":"Unicorn (Vision Safety)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (43 entries)."],["scores.models","pass","41 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 19 months ago, in April 2025, for o3. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","info","Saturated. The best is 91.2%, past 85% since January 2024."],["import.importer","pass","We read the board on VHELM: 46 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"unidentified-video-objects","name":"Unidentified Video Objects","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uniform-bar-exam","name":"Uniform Bar Exam (MBE+MEE+MPT)","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 3 years ago, in March 2023, for GPT-4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"unigenbench","name":"UniGenBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (63 entries) and system cards (1 score)."],["scores.models","pass","16 models we track, plus 47 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Cosmos3-Nano."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 16 rows for models we track and 47 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"unigui-bench","name":"UniGUI-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","11 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GLM-5V-Turbo."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 80.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 11 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"external-agent-messages-openai","name":"Unintended Engagement with External Agent Messages","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (8 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uniqueness-of-optimum","name":"Uniqueness of Optimum","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for gpt-oss-120b."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is gpt-oss-120b (#126)."],["scores.headroom","pass","The best is 55.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 1 row for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"unireditbench","name":"UniREditBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","4 models we track, plus 11 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on maplebb.github.io: 4 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"unit-commitment-lite","name":"Unit Commitment Lite","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"unitybench-mini3d","name":"UnityBench-Mini3D","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (21 entries)."],["scores.models","pass","20 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 72.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on unitybench.github.io: 35 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ud-sanskrit-vedic","name":"Universal Dependencies Sanskrit Vedic","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"universebench","name":"UniVerseBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","5 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Qwen3.5-Omni-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 74.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on 143.89.224.8: 5 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"unjustified-refusals-google","name":"Unjustified refusals","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Argon."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uno-bench","name":"UNO-Bench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","7 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 76.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on meituan-longcat.github.io: 7 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"unsatz","name":"Unsatz","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 60%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"mohler-short-answer-grading-2","name":"UNT Computer Science Short Answer Dataset 2.0","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uq-unsolved-questions","name":"UQ","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"urbansed","name":"UrbanSED","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"urdu-language-speech-emotional-corpus","name":"Urdu Language Speech Emotional Corpus","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aaivu.github.io: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"urduser","name":"UrduSER","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aaivu.github.io: 11 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"urgent-2024","name":"URGENT 2024","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uro-bench","name":"URO-Bench","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen3.5-Omni-Plus."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 84.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dermmcqa-google","name":"US-DermMCQA (Google)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"usabo-semifinal-2020","name":"USABO Semifinal Exam 2020","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 3 years ago, in March 2023, for GPT-4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"usaco","name":"USACO","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for Claude 3 Sonnet."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 Turbo (#176)."],["scores.headroom","pass","The best is 20.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on princeton-nlp.github.io: 17 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"usamo-2024","name":"USAMO 2024","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"usamo-2025","name":"USAMO 2025","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries) and launch posts (2 scores)."],["scores.models","pass","12 models we track."],["scores.recent","info","The newest came 16 months ago, in July 2025, for Grok 4 Heavy. That’s expected for a test replaced by USAMO 2026."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Grok 4 (#86)."],["scores.headroom","pass","The best is 61.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on matharena.ai: 10 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"usamo-2026","name":"USAMO 2026","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries), launch posts (3 scores) and system cards (5 scores)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from July 2026, for Tencent Hy3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Saturated. The best is 98.2%, past 85% since March 2026."],["import.importer","pass","We read the board on MathArena: 9 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"computer-use-confirmations-openai","name":"User Confirmations During Computer Use","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 92%, past 85% since January 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"userbench-salesforce","name":"UserBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","9 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 18 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"userbench-developer-simulation","name":"UserBench (developer simulation)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","pass","The best is 51.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on userbench.vercel.app: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"usim-underwater-robot-tasks","name":"USIM underwater robot tasks","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for GR00T N1.5."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 37.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"usmle-step-1","name":"USMLE Step 1 (sample questions)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Claude 2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 2 (#192)."],["scores.headroom","pass","The best is 68.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"usmle-step-2","name":"USMLE Step 2 (sample questions)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Claude 2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 2 (#192)."],["scores.headroom","pass","The best is 63.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"usmle-step-3","name":"USMLE Step 3 (sample questions)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 3 years ago, in July 2023, for Claude 2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 2 (#192)."],["scores.headroom","pass","The best is 67.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"usnco-local-section-2022","name":"USNCO Local Section Exam 2022","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 3 years ago, in March 2023, for GPT-4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4 (#181)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uw-section-chunker-i2b2-discharge","name":"UW Clinical Record Section Chunker i2b2 Discharge Summary Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uwbench","name":"UWBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uyghur-language-question-answering","name":"Uyghur Language Question Answering","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uzbek-open-asr-leaderboard","name":"Uzbek Open ASR Leaderboard","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","34 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 34 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"uzbek-speech-corpus","name":"Uzbek Speech Corpus","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","34 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on NavAI: 34 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"v","name":"V*","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries), launch posts (15 scores) and system cards (5 scores)."],["scores.models","pass","19 models we track, plus 11 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K2.6 (#63)."],["scores.headroom","info","Saturated. The best is 96.9%, past 85% since May 2025."],["import.importer","pass","We read the board on vstar-seal.github.io: 2 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"v-jepa-2-zero-shot-robot-manipulation-meta","name":"V-JEPA 2 zero-shot robot manipulation","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vaani-benchmark-v1","name":"Vaani Benchmark V1.0","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","warn","The record links a board on huggingface.co that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vabar","name":"VABAR","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vaers-influenza-gbs","name":"VAERS influenza vaccine adverse event extraction","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vaguebench","name":"VagueBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for InternVL3-78B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 43.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vahbench","name":"VAHBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (32 entries)."],["scores.models","pass","1 model we track, plus 31 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 30 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vakra","name":"VAKRA","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Qwen 3.6 35B-A3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 38.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ibm-research-vakra.hf.space: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"valerius-flaccus-intertexts","name":"Valerius Flaccus Intertextuality Benchmark","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"valor-32k","name":"VALOR-32K","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-finance-agent","name":"Vals Finance Agent","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (49 entries), launch posts (7 scores) and system cards (5 scores)."],["scores.models","pass","52 models we track."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 Pro (#19)."],["scores.headroom","pass","The best is 64.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 51 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 5 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-finance-agent-v1","name":"Vals Finance Agent v1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (47 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","47 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Opus 4.6. That’s expected for a test replaced by Vals Finance Agent."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 60.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 59 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-finance-agent-v2","name":"Vals Finance Agent v2","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (80 entries), launch posts (10 scores) and system cards (9 scores)."],["scores.models","pass","82 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 65.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 81 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 9 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-finance-agent-v2-1","name":"Vals Finance Agent v2.1","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","45 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 69.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 44 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-finance-index","name":"Vals Finance Index","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries), launch posts (2 scores) and system cards (1 score)."],["scores.models","pass","45 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 68.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 44 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-index-v1","name":"Vals Index v1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (35 entries) and system cards (1 score)."],["scores.models","pass","35 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 74.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 34 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-index-v2","name":"Vals Index v2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (65 entries)."],["scores.models","pass","65 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 69.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 66 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-index-v2-1","name":"Vals Index v2.1","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (49 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","49 models we track."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 68.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 50 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-index","name":"Vals Index v2.2","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (42 entries), launch posts (4 scores) and system cards (3 scores)."],["scores.models","pass","42 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 71.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 41 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-inference-provider-index","name":"Vals Inference Provider Index","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 71.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vlair-vals","name":"Vals Legal AI Report","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-legal-index","name":"Vals Legal Index","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (43 entries) and launch posts (1 score)."],["scores.models","pass","42 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 78.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 42 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-multimodal-index","name":"Vals Multimodal Index","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries)."],["scores.models","pass","33 models we track."],["scores.recent","pass","The newest is from August 2026, for Muse Spark 1.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 74.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 33 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-multimodal-index-v1","name":"Vals Multimodal Index v1","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","28 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Grok 4.3. That’s expected for a test replaced by Vals Multimodal Index v1.1."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 71.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 28 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-multimodal-index-v1-1","name":"Vals Multimodal Index v1.1","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Claude Opus 4.8. That’s expected for a test replaced by Vals Multimodal Index."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 70.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 15 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-router-index","name":"Vals Router Index","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on vals.ai: 6 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-rsi-index","name":"Vals RSI Index","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","28 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","8 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on vals.ai: 28 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-science-index","name":"Vals Science Index","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries) and launch posts (2 scores)."],["scores.models","pass","28 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 68.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 28 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-abhishek203-sfagent","name":"Vals Smith: abhishek203/SfAgent","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-apache-spark","name":"Vals Smith: apache/spark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 84.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-arandomhooman-hoomans-morphe-patches","name":"Vals Smith: arandomhooman/hoomans-morphe-patches","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 20%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 4 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-arduino-arduino-ide","name":"Vals Smith: arduino/arduino-ide","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.6 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Luna (#36)."],["scores.headroom","pass","The best is 43.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 3 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-astral-sh-uv","name":"Vals Smith: astral-sh/uv","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 66%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-aumeshmisra-tensorzero","name":"Vals Smith: AumeshMisra/tensorzero","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 86.7%, past 85% since July 2026."],["import.importer","pass","We read the board on Vals AI: 3 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-benredmond-apex","name":"Vals Smith: benredmond/apex","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Luna."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.6 Terra (#18)."],["scores.headroom","pass","The best is 63.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-berriai-litellm","name":"Vals Smith: BerriAI/litellm","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.6 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.6 Flash (#45)."],["scores.headroom","pass","The best is 70%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-chrisanderton-tyk-gateway","name":"Vals Smith: chrisanderton/tyk-gateway","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 76.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 4 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-conjfrnk-blackbird","name":"Vals Smith: conjfrnk/blackbird","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from August 2026, for Muse Spark 1.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 56.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 3 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-conjfrnk-memovox","name":"Vals Smith: conjfrnk/memovox","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 90%, past 85% since July 2026."],["import.importer","pass","We read the board on Vals AI: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-dwojtaszek-zipper","name":"Vals Smith: dwojtaszek/zipper","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 93.3%, past 85% since July 2026."],["import.importer","pass","We read the board on Vals AI: 3 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-effect-ts-effect","name":"Vals Smith: Effect-TS/effect","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 93.3%, past 85% since July 2026."],["import.importer","pass","We read the board on Vals AI: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-expertium-fsrs-autoresearch","name":"Vals Smith: Expertium/fsrs-autoresearch","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 96.7%, past 85% since May 2026."],["import.importer","pass","We read the board on Vals AI: 4 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-flutter-flutter","name":"Vals Smith: flutter/flutter","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.6 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.6 Flash (#45)."],["scores.headroom","pass","The best is 50%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-fmtlib-fmt","name":"Vals Smith: fmtlib/fmt","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from August 2026, for Muse Spark 1.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.2 (#46)."],["scores.headroom","pass","The best is 83.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 3 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-geopandas-geopandas","name":"Vals Smith: geopandas/geopandas","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for Gemini 3.6 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 66.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 4 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-goharbor-harbor","name":"Vals Smith: goharbor/harbor","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 60%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 4 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-jax-ml-jax","name":"Vals Smith: jax-ml/jax","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 79.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-ladydascalie-core","name":"Vals Smith: ladydascalie/core","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.6 Flash (#45)."],["scores.headroom","pass","The best is 76.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 4 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-leanprover-community-mathlib4","name":"Vals Smith: leanprover-community/mathlib4","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from August 2026, for Muse Spark 1.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 76.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 4 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-lemonawa-music163bot-rust","name":"Vals Smith: Lemonawa/music163bot-rust","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 93.3%, past 85% since July 2026."],["import.importer","pass","We read the board on Vals AI: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-mirumee-ariadne","name":"Vals Smith: mirumee/ariadne","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek V4 Flash 0731 (#52)."],["scores.headroom","pass","The best is 83.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-nationalsecurityagency-ghidra","name":"Vals Smith: nationalsecurityagency/ghidra","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Grok 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4.5 (#47)."],["scores.headroom","pass","The best is 73.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-nvidia-nccl","name":"Vals Smith: NVIDIA/nccl","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from August 2026, for Muse Spark 1.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 46.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 3 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-openttd-openttd","name":"Vals Smith: OpenTTD/OpenTTD","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since July 2026."],["import.importer","pass","We read the board on Vals AI: 3 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-oven-sh-bun","name":"Vals Smith: oven-sh/bun","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 54.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-palantir-blueprint","name":"Vals Smith: palantir/blueprint","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Saturated. The best is 93.2%, past 85% since April 2026."],["import.importer","pass","We read the board on Vals AI: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-pallets-eco-flask-wtf","name":"Vals Smith: pallets-eco/flask-wtf","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since July 2026."],["import.importer","pass","We read the board on Vals AI: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-pingdotgg-t3code","name":"Vals Smith: pingdotgg/t3code","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","An importer exists, but it hasn’t read the board yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-poalrom-sharepaste","name":"Vals Smith: poalrom/sharepaste","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 50%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-punt-labs-lux","name":"Vals Smith: punt-labs/lux","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from August 2026, for Muse Spark 1.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 73.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 3 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-sashabaranov-go-openai","name":"Vals Smith: sashabaranov/go-openai","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 96.7%, past 85% since July 2026."],["import.importer","pass","We read the board on Vals AI: 5 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-schubydoo-clauster","name":"Vals Smith: schubydoo/clauster","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","pass","The newest is from August 2026, for Muse Spark 1.2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 96.7%, past 85% since July 2026."],["import.importer","pass","We read the board on Vals AI: 6 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-sgl-project-sglang","name":"Vals Smith: sgl-project/sglang","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 78%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-skypilot-org-skypilot","name":"Vals Smith: skypilot-org/skypilot","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 30%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 4 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-suitedaces-dorabot","name":"Vals Smith: suitedaces/dorabot","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from August 2026, for Muse Spark 1.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.8 (#22)."],["scores.headroom","pass","The best is 66.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 4 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-tensorflow-tensorflow","name":"Vals Smith: tensorflow/tensorflow","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from August 2026, for Muse Spark 1.2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 90%, past 85% since October 2025."],["import.importer","pass","We read the board on Vals AI: 5 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-tock-tock","name":"Vals Smith: tock/tock","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for DeepSeek V4 Flash 0731."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Sonnet 5 (#35)."],["scores.headroom","info","Saturated. The best is 86.7%, past 85% since June 2026."],["import.importer","pass","We read the board on Vals AI: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-torvalds-linux","name":"Vals Smith: torvalds/linux","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Muse Spark 1.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 74.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-vastsa-pi-desktop","name":"Vals Smith: vastsa/PI-Desktop","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from August 2026, for Muse Spark 1.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Muse Spark 1.2 (#46)."],["scores.headroom","pass","The best is 40%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 3 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-vercel-next-js","name":"Vals Smith: vercel/next.js","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for Kimi K3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 74.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 13 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-smith-vllm-project-llm-compressor","name":"Vals Smith: vllm-project/llm-compressor","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Saturated. The best is 96.7%, past 85% since July 2026."],["import.importer","pass","We read the board on Vals AI: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-web-search-index","name":"Vals Web Search Index","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 57.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 12 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"valyrian-games-coding-challenge","name":"Valyrian Games Coding Challenge","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","17 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5 mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 mini (#92)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on valyriantech.github.io: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vamos-bench","name":"VAmoS Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","1 model we track, plus 16 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Gemini 3.1 Flash Live. That’s expected for a test replaced by VAmoS Pro Bench."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on veris.ai: 14 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vamos-pro-bench","name":"VAmoS Pro Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","3 models we track, plus 17 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Live."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 40.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on veris.ai: 13 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vantage-bench","name":"VANTAGE-BENCH","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","14 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 14 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"varex-bench","name":"VAREX-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","8 models we track, plus 12 other entries on its leaderboard."],["scores.recent","info","The newest came 11 months ago, in December 2025, for Ministral 3 14B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 2.5 Flash (#112)."],["scores.headroom","info","Saturated. The best is 97.3%, past 85% since April 2025."],["import.importer","pass","We read the board on udibarzi.github.io: 8 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"variantbench","name":"VariantBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 52.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on benchmarks.bio: 42 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vatex","name":"VATEX","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","5 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini Embedding 2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eric-xw.github.io: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vatex-zh","name":"VATEX ZH","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","4 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on eric-xw.github.io: 7 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vbench","name":"VBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (74 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","5 models we track, plus 70 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for ERNIE 5.0. That’s expected for a test replaced by VBench-2.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 85.1%, past 85% since August 2025."],["import.importer","pass","We read the board on huggingface.co: 4 rows for models we track and 70 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vbench-i2v","name":"VBench image-to-video","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","1 model we track, plus 33 other entries on its leaderboard."],["scores.recent","info","The newest came 21 months ago, in February 2025, for Wan2.1-I2V-14B-720P. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 86.9%, past 85% since February 2025."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 33 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vbench-2-0","name":"VBench-2.0","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","1 model we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Veo 3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 66.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 13 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vbvr-bench","name":"VBVR-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries)."],["scores.models","pass","9 models we track, plus 21 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Wan2.2-I2V-A14B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on video-reason.com: 9 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vbvr-pro","name":"VBVR-Pro","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (30 entries)."],["scores.models","pass","9 models we track, plus 21 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Wan2.2-I2V-A14B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on video-reason.com: 9 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vcb-bench","name":"VCB-Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vcbench","name":"VCBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (31 entries)."],["scores.models","pass","10 models we track, plus 21 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 33.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vcbench.com: 10 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vcg-diverse","name":"VCG+ Diverse","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","1 model we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for GPT-4V."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vcis","name":"VCIS","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Sonnet 5.5."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Sonnet 5.5 (#9)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since April 2026."],["import.importer","pass","We read the board on kaggle.com: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vcr-visual-caption-restoration","name":"VCR (Visual Caption Restoration)","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (48 entries) and system cards (3 scores)."],["scores.models","pass","45 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Nemotron Nano 12B v2 VL. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","info","Saturated. The best is 97%, past 85% since August 2025."],["import.importer","pass","We read the board on huggingface.co: 42 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vcr-bench","name":"VCR-Bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","10 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Seed1.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1 (#118)."],["scores.headroom","pass","The best is 59.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vlm-reasoning.github.io: 9 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vcsum","name":"VCSUM","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 3 years ago, in September 2023, for Qwen-14B. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen-14B (#214)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vct-v2-securebio","name":"VCT-v2","summary":{"pass":12,"warn":0,"fail":0,"info":0,"text":"All 12 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries) and system cards (1 score)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 60.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on securebio.org: 27 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vctk-corpus","name":"VCTK Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vcx-benchmark","name":"vCX Benchmark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (44 entries)."],["scores.models","pass","43 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","6 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 93.3%, past 85% since August 2025."],["import.importer","pass","We read the board on bench.retellai.com: 60 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vdc-benchmark","name":"VDC Benchmark","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","6 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for LLaVA-OneVision 7B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vdebench","name":"VDEBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Qwen-Image-Edit-2509."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on raw.githubusercontent.com: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vdr-bench","name":"VDR-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","8 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Qwen3-VL-8B-Instruct."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 27.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on raw.githubusercontent.com: 16 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vectara-hallucination-leaderboard-2","name":"Vectara Hallucination Leaderboard (2025 dataset)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (105 entries)."],["scores.models","pass","104 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 110 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vectara-hallucination-leaderboard-hhem","name":"Vectara Hallucination Leaderboard (HHEM)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (166 entries) and system cards (1 score)."],["scores.models","pass","148 models we track, plus 18 other entries on its leaderboard."],["scores.recent","info","The newest came 13 months ago, in October 2025, for Claude Haiku 4.5. That’s expected for a test replaced by Vectara Hallucination Leaderboard (2025 dataset)."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 154 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"velociti","name":"VELOCITI","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for Qwen2-VL-7B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 43.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on katha-ai.github.io: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vending-bench","name":"Vending-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries) and launch posts (1 score)."],["scores.models","pass","9 models we track."],["scores.recent","info","The newest came 12 months ago, in November 2025, for Gemini 3 Pro. That’s expected for a test replaced by Vending-Bench 2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on andonlabs.com: 9 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vending-bench-2","name":"Vending-Bench 2","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (70 entries), launch posts (6 scores) and system cards (8 scores)."],["scores.models","pass","70 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","9 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on andonlabs.com: 77 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 12 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vending-bench-arena","name":"Vending-Bench Arena","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (32 entries)."],["scores.models","pass","32 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","5 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on andonlabs.com: 48 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"venusbench-mobile","name":"VenusBench-Mobile","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","2 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Kimi K2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K2.5 (#76)."],["scores.headroom","pass","The best is 24.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ui-venus.github.io: 5 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vera-mh","name":"VERA-MH Suicide Risk","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","28 models we track."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on vera-mh.com: 28 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"verbalized-grader-awareness-anthropic","name":"Verbalized grader awareness (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"verbalized-metagaming-openai","name":"Verbalized metagaming","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"verifiability-judgments-for-generative-search","name":"Verifiability Judgments for Generative Search","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"verifiable-browsing-task-meta","name":"Verifiable Browsing Task (Meta)","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"verifierbench-opencompass","name":"VerifierBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (19 entries)."],["scores.models","pass","9 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Qwen3-14B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3-235B-A22B (#131)."],["scores.headroom","pass","The best is 81.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"verify-gym","name":"Verify Gym","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek V4.1 Flash (#34)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 7 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"verifying-the-verifiers-project-ape-cred-benchmark","name":"Verifying the Verifiers (Project APE CRED benchmark)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","23 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Luna."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 99.5%, past 85% since April 2026."],["import.importer","pass","We read the board on ape.socialcatalystlab.org: 45 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"verigap","name":"VeriGap","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","4 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"verigen-design-problems","name":"VeriGen design problems","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries)."],["scores.models","pass","27 models we track, plus 18 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for GLM-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GLM-5.2 (#55)."],["scores.headroom","pass","The best is 77.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on TuRTLe: 27 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"verilogeval","name":"VerilogEval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (83 entries)."],["scores.models","pass","2 models we track, plus 81 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for o1-preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-preview (#149)."],["scores.headroom","pass","The best is 77.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Chip Design LLM Zoo: 3 rows for models we track and 80 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"verilogeval-2","name":"VerilogEval v2","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (44 entries)."],["scores.models","pass","19 models we track, plus 25 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for GLM-5.2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GLM-5.2 (#55)."],["scores.headroom","info","Saturated. The best is 88%, past 85% since January 2026."],["import.importer","pass","We read the board on TuRTLe: 20 rows for models we track and 25 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"veris-finagent-arena-1-1","name":"Veris FinAgent Arena v1.1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Hydrogen."],["scores.frontier","pass","5 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on veris.ai: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vero-bench","name":"VeRO-Bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on labs.scale.com: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"verticalrtk","name":"VerticalRTK","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on tako.com: 7 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"verus-bench","name":"Verus-Bench","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on microsoft.github.io: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"verusage-bench","name":"VeruSAGE-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on microsoft.github.io: 10 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ves-label-check","name":"VES Label Check","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 5 (#35)."],["scores.headroom","pass","The best is 14.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on kaggle.com: 2 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"veteval","name":"VetEval","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","29 models we track."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an index, with no fixed ceiling."],["import.importer","pass","We read the board on veteval.ai: 29 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vggsound","name":"VGGSound","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vgi-bench","name":"VGI-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","7 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on hexuan21.github.io: 7 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vhelm","name":"VHELM","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (43 entries)."],["scores.models","pass","41 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on crfm.stanford.edu: 46 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vials","name":"VIALS","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 65.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vials-2","name":"VIALS 2","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 49.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on joinhandshake.com: 14 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vibe-minimax","name":"VIBE (MiniMax)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and launch posts (1 score)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 11 months ago, in December 2025, for MiniMax-M2.1. That’s expected for a test replaced by VIBE-Pro (MiniMax)."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Saturated. The best is 90.7%, past 85% since September 2025."],["import.importer","pass","We read the board on github.com: 5 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vibe-code-bench","name":"Vibe Code Bench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (126 entries), launch posts (3 scores) and system cards (2 scores)."],["scores.models","pass","125 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Saturated. The best is 93.8%, past 85% since June 2026."],["import.importer","pass","We read the board on vals.ai: 131 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-vibe-code-bench-1-100","name":"Vibe Code Bench 1-100","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","26 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 34%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 26 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-vibe-code-bench-1-100-v1-1","name":"Vibe Code Bench 1-100 v1.1","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","pass","The best is 55.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vals-vibe-code-bench-v1","name":"Vibe Code Bench v1","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. That’s expected for a test replaced by Vibe Code Bench."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 41.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vals.ai: 17 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vibe-eval","name":"Vibe-Eval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries), launch posts (8 scores) and system cards (6 scores)."],["scores.models","pass","16 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Seed2.0 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 81.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vibe-pro-minimax","name":"VIBE-Pro (MiniMax)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores) and system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for MiniMax-M2.5. That’s expected for a test replaced by VIBE-V2 (MiniMax)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 55.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vibe-v2-minimax","name":"VIBE-V2 (MiniMax)","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is MiniMax-M3 (#80)."],["scores.headroom","pass","The best is 50.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vibench-2","name":"ViBench 2.0","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 64.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vibench.ai: 9 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vibeprevisbench","name":"VibePrevisBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","11 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on vibeprevisbench.github.io: 11 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vibevideobench","name":"VibeVideoBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","7 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on vibevideobench.github.io: 9 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vbs-2026-marine-video-kit","name":"Video Browser Showdown 2026 marine video search","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video-object-segmentation-under-transformations","name":"Video Object Segmentation under Transformations","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on vostdataset.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video-reality-test","name":"Video Reality Test","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","6 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 76.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on video-reality-test.github.io: 6 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video-simpleqa","name":"Video SimpleQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (38 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","31 models we track, plus 11 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 76.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on videosimpleqa.github.io: 27 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video-to-valence","name":"Video to Valence","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video-chatgpt-video-conversation-benchmark","name":"Video-ChatGPT video conversation benchmark","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video-holmes","name":"Video-Holmes","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries), launch posts (4 scores) and system cards (4 scores)."],["scores.models","pass","12 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","pass","The best is 68.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on video-holmes.github.io: 7 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video-index","name":"Video-Index","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for Qwen4-Max."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 68.1%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video-mme","name":"Video-MME","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (51 entries), launch posts (47 scores) and system cards (27 scores)."],["scores.models","pass","77 models we track, plus 25 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for LongCat-2.5."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K3 (#25)."],["scores.headroom","info","Saturated. The best is 90.4%, past 85% since June 2025."],["import.importer","pass","We read the board on video-mme.github.io: 52 rows for models we track and 25 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video-mmmu","name":"Video-MMMU","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries), launch posts (33 scores) and system cards (16 scores)."],["scores.models","pass","56 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","info","Saturated. The best is 88.7%, past 85% since November 2025."],["import.importer","pass","We read the board on videommmu.github.io: 24 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 4 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video-monetbench","name":"Video-MonetBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video-oasis","name":"Video-Oasis","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","6 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Qwen3-VL-8B-Instruct."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 34.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on limgeuntaekk.github.io: 6 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video-safetybench","name":"Video-SafetyBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (25 entries)."],["scores.models","pass","15 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Qwen2.5-VL-32B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 30 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"video2robot-embodied","name":"Video2Robot Embodied","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videoace","name":"VideoACE","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 4 Argon (#10)."],["scores.headroom","info","Saturated. The best is 88.1%, past 85% since September 2026."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videocon","name":"VideoCon","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videodr-with-search","name":"VideoDR (with search)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (1 score)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 76%, under 85%, so it still separates models."],["import.importer","pass","We read the board on videodr-benchmark.github.io: 12 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videoeval-pro","name":"VideoEval-Pro","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","12 models we track, plus 18 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Seed2.0 Lite (0428)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4.1 (#138)."],["scores.headroom","pass","The best is 49.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tiger-ai-lab.github.io: 10 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videofdb-nvidia","name":"VideoFDB","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","5 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Gemini 3.1 Flash Live."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on research.nvidia.com: 5 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videogaia","name":"VideoGAIA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries)."],["scores.models","pass","20 models we track."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 58.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 20 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videogamebench","name":"VideoGameBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Llama 4 Maverick."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#104)."],["scores.headroom","pass","The best is 0.48%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vgbench.com: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videogamebench-lite","name":"VideoGameBench Lite","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for Llama 4 Maverick."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#104)."],["scores.headroom","pass","The best is 1.60%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vgbench.com: 5 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videogen-rewardbench","name":"VideoGen-RewardBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videohallucer","name":"VideoHallucer","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 16 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Reka Edge."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 59.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 16 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videomme-v2-with-subtitles","name":"VideoMME v2 (with subtitles)","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (37 entries), launch posts (6 scores) and system cards (3 scores)."],["scores.models","pass","39 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 68.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on video-mme-v2.netlify.app: 50 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videophy","name":"VideoPhy","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 12 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videophy-2","name":"VideoPhy-2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries) and system cards (1 score)."],["scores.models","pass","6 models we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Cosmos3-Nano."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 40.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on videophy2.github.io: 5 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videoreasonbench","name":"VideoReasonBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","17 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Seed2.0 Lite (0428)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o4-mini (#95)."],["scores.headroom","pass","The best is 77.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on llyx97.github.io: 15 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videoscore-bench","name":"VideoScore-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","6 models we track, plus 11 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in May 2024, for Gemini 1.5 Flash. That’s expected for a test replaced by VideoScore-Bench v2."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4o (#162)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on tiger-ai-lab.github.io: 6 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"videoscore-bench-2","name":"VideoScore-Bench v2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","3 models we track, plus 13 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 28.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vidore-v2","name":"ViDoRe v2","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 6 months ago, in May 2026, for Gemini Embedding 2. That’s expected for a test replaced by ViDoRe V3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vidore-v3","name":"ViDoRe V3","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (53 entries)."],["scores.models","pass","2 models we track, plus 51 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for Qwen3-VL-Embedding-2B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mteb-leaderboard.hf.space: 2 rows for models we track and 51 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vidscribe","name":"VidScribe","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"viewspatial-bench","name":"ViewSpatial-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","11 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.0 Flash (#147)."],["scores.headroom","pass","The best is 61.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vif-bench","name":"VIF-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Qwen-Image-Edit-2509."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vigil3d","name":"ViGiL3D","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on codabench.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vilegalexpert","name":"ViLegalExpert","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"villm-eval","name":"ViLLM-Eval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vima-bench","name":"VIMA-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (28 entries)."],["scores.models","pass","28 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 28 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vimu","name":"ViMU","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for MiMo-V2-Omni."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 46.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on liqiiiii.github.io: 16 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vinoground","name":"Vinoground","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (23 entries)."],["scores.models","pass","23 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 23 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vinv-tts","name":"ViNV-TTS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vipbench","name":"VIPBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vipseg","name":"VIPSeg","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"virology-capabilities-test-text-only-securebio","name":"Virology Capabilities Test (text-only)","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 46%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"virology-capabilities-test-vct","name":"Virology Capabilities Test (VCT)","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries), launch posts (1 score) and system cards (32 scores)."],["scores.models","pass","38 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Spark 1.3."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 67.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on securebio.org: 7 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"virtual-kitti-2","name":"Virtual KITTI 2","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on europe.naverlabs.com: 1 entry at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"viseditbench-tikz","name":"VisEditBench (TikZ diagram editing)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries)."],["scores.models","pass","25 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 31B IT."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 83.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on viseditbench.github.io: 29 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"visfactor","name":"VisFactor","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","31 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 60.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 27 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vision-arena","name":"Vision Arena","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (153 entries)."],["scores.models","pass","138 models we track, plus 15 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arena.ai: 152 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vision2web","name":"Vision2Web","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries), launch posts (7 scores) and system cards (1 score)."],["scores.models","pass","16 models we track."],["scores.recent","pass","The newest is from November 2026, for LongCat-2.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 82.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vision2web-bench.github.io: 10 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vision2web-s1-2026","name":"Vision2Web S1-2026","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","5 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 11 months ago, in December 2025, for Gemini 3 Flash. That’s expected for a test replaced by Vision2Web S2-2026."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 52.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vision2web-bench.github.io: 10 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vision2web-s2-2026","name":"Vision2Web S2-2026","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for MiMo-V2-Omni. That’s expected for a test replaced by Vision2Web S3-2026."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 43.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vision2web-bench.github.io: 9 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vision2web-s3-2026","name":"Vision2Web S3-2026","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from June 2026, for MiniMax-M3."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 74.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vision2web-bench.github.io: 7 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"visionarena-bench","name":"VisionArena-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","16 models we track."],["scores.recent","warn","The newest came 2 years ago, in October 2024, for Gemini 1.5 Flash 8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 32 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vispeak-bench","name":"ViSpeak-Bench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","9 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Seed2.0 Lite (0428). That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 1.5 Pro (#157)."],["scores.headroom","info","Saturated. The best is 87%, past 85% since April 2026."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"visr-rag-fin","name":"VisR-RAG-Fin","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vist-toloka","name":"VIST","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vista-scale","name":"VISTA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (54 entries)."],["scores.models","pass","54 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 Pro (#19)."],["scores.headroom","pass","The best is 54.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 63 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vista-blind-low-vision","name":"VISTA (Visually Impaired Scene and Task Annotations)","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","11 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Gemini 3.1 Flash-Lite."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Flash-Lite (#96)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on harvard-ai-and-robotics-lab.github.io: 11 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vistaar","name":"Vistaar","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vistorybench","name":"ViStoryBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","1 model we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Qwen-Image-Edit-2509."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on vistorybench.github.io: 1 row for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vistra","name":"Vistra","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores) and system cards (1 score)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for TranslateGemma 27B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"visual-aesthetic-benchmark","name":"Visual Aesthetic Benchmark","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","19 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 40.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vab.bakelab.ai: 19 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"fab-visual-reasoning","name":"Visual Reasoning Benchmark (Fab AI)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (59 entries)."],["scores.models","pass","57 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Saturated. The best is 89.9%, past 85% since May 2026."],["import.importer","pass","We read the board on fab-ai.org: 65 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"visual-strategic-bench","name":"Visual Strategic Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","9 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for o3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 84.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vs-bench.github.io: 10 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"visualagentbench","name":"VisualAgentBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries)."],["scores.models","pass","5 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for GPT-4o mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 36.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 5 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"visualprobe","name":"VisualProbe","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Step 3.7 Flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 65%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 6 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"visualpuzzles","name":"VisualPuzzles","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","32 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for ERNIE 5.0."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 64.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on neulab.github.io: 31 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"visualtoolbench","name":"VisualToolBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from July 2026, for Muse Spark 1.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 44.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on labs.scale.com: 21 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"visualwebarena","name":"VisualWebArena","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","7 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Llama 3-70B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-70B (#182)."],["scores.headroom","pass","The best is 16.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on jykoh.com: 15 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"visualwebbench","name":"VisualWebBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (22 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","9 models we track, plus 15 other entries on its leaderboard."],["scores.recent","info","The newest came 13 months ago, in October 2025, for LongCat-Flash-Omni. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 87.3%, past 85% since May 2025."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"visulogic","name":"VisuLogic","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (25 entries), launch posts (5 scores) and system cards (6 scores)."],["scores.models","pass","22 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 54.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on visulogic-benchmark.github.io: 17 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vita-bench","name":"VITA-Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries), launch posts (16 scores) and system cards (5 scores)."],["scores.models","pass","32 models we track."],["scores.recent","pass","The newest is from June 2026, for Keye-VL-2.0-30B-A3B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.7-Max (#49)."],["scores.headroom","pass","The best is 49.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vitabench.github.io: 26 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vitabench-2","name":"VitaBench 2.0","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries) and launch posts (1 score)."],["scores.models","pass","18 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Spark-X2.6-4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 50.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vitabench2.github.io: 72 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vitabench-noise-longcat","name":"VitaBench-Noise (LongCat)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 10 months ago, in January 2026, for LongCat-Flash-Thinking-2601. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 20.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"viverbench","name":"ViVerBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries) and system cards (1 score)."],["scores.models","pass","9 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Seed2.0 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 75.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on omniverifier.github.io: 8 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vizwiz-vqa","name":"VizWiz","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vl-rewardbench","name":"VL-RewardBench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (16 entries)."],["scores.models","pass","8 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for NVLM-D-72B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.2 90B (#179)."],["scores.headroom","pass","The best is 53.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vl-rewardbench.github.io: 8 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vla-arena","name":"VLA-Arena","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (28 entries)."],["scores.models","pass","8 models we track, plus 20 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for MolmoAct 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 72.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vla-arena.github.io: 10 rows for models we track and 20 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vlabench","name":"VLABench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in June 2024, for OpenVLA."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 9.84%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vladbench","name":"VLADBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (25 entries) and launch posts (1 score)."],["scores.models","pass","8 models we track, plus 18 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Qwen3.7-Plus."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.7-Plus (#79)."],["scores.headroom","pass","The best is 77.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vlair-legal-research-vals","name":"VLAIR Legal Research","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vlegal-bench","name":"VLegal-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (23 entries)."],["scores.models","pass","7 models we track, plus 16 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 4.5 (#81)."],["scores.headroom","pass","The best is 72%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 7 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vlguard","name":"VLGuard","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Phi-4-multimodal."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vlms-are-blind","name":"VLMs Are Blind","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries), launch posts (13 scores) and system cards (5 scores)."],["scores.models","pass","14 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Qwen3.6 27B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.6 27B (#88)."],["scores.headroom","info","Saturated. The best is 98.6%, past 85% since May 2025."],["import.importer","pass","We read the board on vlmsareblind.github.io: 4 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vlmsarebiased","name":"VLMsAreBiased","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries), launch posts (4 scores) and system cards (3 scores)."],["scores.models","pass","10 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 88.3%, past 85% since August 2026."],["import.importer","pass","We read the board on vlmsarebiased.github.io: 17 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vlsp-2023-vietnamese-mispronunciation-detection","name":"VLSP 2023 Vietnamese Mispronunciation Detection","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vm14k","name":"VM14K","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vmax-tasks","name":"Vmax Tasks","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vocalgrad","name":"VocalGrad","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for Gemini 3 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Flash (#60)."],["scores.headroom","pass","The best is 51.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vocalsound","name":"VocalSound","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voice-agent-llm-benchmark-hamming","name":"Voice Agent LLM Benchmark","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicearena-arabic-msa","name":"Voice Arena Arabic MSA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries) and system cards (2 scores)."],["scores.models","pass","7 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash-Lite TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on voicearena.com: 7 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicearena-bahasa","name":"Voice Arena Bahasa","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash-Lite TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on voicearena.com: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicearena-brazilian-portuguese","name":"Voice Arena Brazilian Portuguese","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries) and system cards (2 scores)."],["scores.models","pass","7 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash-Lite TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on voicearena.com: 7 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"diarization-bench-voicearena","name":"Voice Arena Diarization Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","1 model we track, plus 14 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Muse Voice Transcribe 1.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on voicearena.com: 3 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"dictation-bench-voicearena","name":"Voice Arena Dictation Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","1 model we track, plus 13 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.5 Transcribe."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 75.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on voicearena.com: 1 row for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicearena-french","name":"Voice Arena French","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","3 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash-Lite TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on voicearena.com: 3 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"full-duplex-bench-voicearena","name":"Voice Arena Full Duplex Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Live."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 99.9%, past 85% since September 2026."],["import.importer","pass","We read the board on voicearena.com: 3 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicearena-german","name":"Voice Arena German","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","3 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash-Lite TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on voicearena.com: 3 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voice-arena-goal-dataset","name":"Voice Arena Goal Dataset","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 2 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Qwen2.5-Omni-7B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 64.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 2 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicearena-hindi","name":"Voice Arena Hindi","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries) and system cards (2 scores)."],["scores.models","pass","9 models we track, plus 14 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Voxtral Mini TTS 3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on voicearena.com: 9 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicearena-indian-english","name":"Voice Arena Indian English","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","3 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash-Lite TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on voicearena.com: 3 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicearena-japanese","name":"Voice Arena Japanese","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries) and system cards (2 scores)."],["scores.models","pass","6 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash-Lite TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on voicearena.com: 6 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicearena-mexican-spanish","name":"Voice Arena Mexican Spanish","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries) and system cards (2 scores)."],["scores.models","pass","7 models we track, plus 5 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash-Lite TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on voicearena.com: 7 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicearena-tagalog","name":"Voice Arena Tagalog","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash-Lite TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on voicearena.com: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicearena-us-english","name":"Voice Arena US English","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries) and system cards (2 scores)."],["scores.models","pass","8 models we track, plus 18 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Voxtral Mini TTS 3."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on voicearena.com: 8 rows for models we track and 18 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicearena-vietnamese","name":"Voice Arena Vietnamese","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries) and system cards (2 scores)."],["scores.models","pass","7 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash-Lite TTS."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on voicearena.com: 7 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voice-controllability-hume","name":"Voice Controllability Leaderboard","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 6 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voice-jailbreak-attacks-against-gpt-4o","name":"Voice Jailbreak Attacks Against GPT-4o","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voice-persuasion-study-openai","name":"Voice persuasion study on party preferences","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 2 years ago, in August 2024, for GPT-4o. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 5.74%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voice-replication-hume","name":"Voice Replication Leaderboard","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","1 model we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 12 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicebench","name":"VoiceBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (46 entries), launch posts (11 scores) and system cards (9 scores)."],["scores.models","pass","24 models we track, plus 31 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Inkling-Small (#68)."],["scores.headroom","info","Saturated. The best is 93.1%, past 85% since September 2025."],["import.importer","pass","We read the board on matthewcym.github.io: 16 rows for models we track and 31 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicecodebench","name":"VoiceCodeBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","9 models we track, plus 11 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Muse Voice Transcribe 1.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 67.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on Vals AI: 9 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicemos-challenge-2023","name":"VoiceMOS Challenge 2023","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on voicemos-challenge-2023.github.io: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicemos-challenge-2024","name":"VoiceMOS Challenge 2024","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voicemos-challenge-2026","name":"VoiceMOS Challenge 2026","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on docs.google.com: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vox-dub","name":"VOX-DUB","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voxanon","name":"VoxAnon","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voxceleb1","name":"VoxCeleb1","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on robots.ox.ac.uk: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voxceleb2","name":"VoxCeleb2","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","11 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Flash (#114)."],["scores.headroom","pass","The best is 75.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on AHELM: 14 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voxelbench","name":"VoxelBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (57 entries)."],["scores.models","pass","56 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for GPT-6.1 Terra."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on voxelbench.ai: 56 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voxelbench-image","name":"VoxelBench Image","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on voxelbench.ai: 13 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voxelcodebench","name":"VoxelCodeBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Claude Sonnet 4.5. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Saturated. The best is 87.9%, past 85% since August 2025."],["import.importer","pass","We read the board on arxiv.org: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voxlingua107","name":"VoxLingua107","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on Scholar Compass: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voxmem","name":"VoxMem","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","9 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","pass","The best is 38.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 9 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voxparity","name":"VoxParity","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (23 entries)."],["scores.models","pass","14 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for MiMo-V2.6 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 14 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"voxpopuli","name":"VoxPopuli","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vpct","name":"VPCT","summary":{"pass":5,"warn":1,"fail":0,"info":4,"text":"5 of 6 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from boards Epoch AI copied (27 models), launch posts (1 score), system cards (2 scores) and claims Epoch AI collected (1 model)."],["scores.models","pass","30 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Seed1.8. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Saturated. The best is 91%, past 85% since November 2025."],["import.importer","warn","The record links a board on epoch.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vqa-rad","name":"VQA-RAD","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vqav2","name":"VQAv2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (560 entries), launch posts (6 scores) and system cards (3 scores)."],["scores.models","pass","10 models we track, plus 558 other entries on its leaderboard."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Reka Edge. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","info","Saturated. The best is 88.4%, past 85% since September 2024."],["import.importer","pass","We read the board on eval.ai: 2 rows for models we track and 558 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vrbench","name":"VRBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","19 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Keye-VL-8B-Preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.7 Sonnet (#119)."],["scores.headroom","pass","The best is 68.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vrbench.github.io: 20 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vrsbench","name":"VRSBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for GPT-4V."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 65.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 4 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vsas-bench","name":"VSAS-Bench","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vsec","name":"VSEC","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vsi-bench","name":"VSI-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries), launch posts (2 scores) and system cards (6 scores)."],["scores.models","pass","16 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 69.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vsi-bench-debiased","name":"VSI-Bench-Debiased","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","4 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 19 months ago, in April 2025, for InternVL3-9B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 41.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on vision-x-nyu.github.io: 4 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vsi-super","name":"VSI-SUPER","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"visual-spatial-reasoning","name":"VSR","summary":{"pass":6,"warn":1,"fail":0,"info":5,"text":"6 of 7 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"visual-spatial-reasoning-random","name":"VSR Random Split","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"visual-spatial-reasoning-zero-shot","name":"VSR Zero-Shot Split","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vsti-bench","name":"VSTI-Bench","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 67.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vtr-bench","name":"VTR-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (11 entries)."],["scores.models","pass","5 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for LingBot-Video-MoE."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vulnerability-finding-evaluations-anthropic","name":"Vulnerability finding evaluations (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vulngym","name":"VulnGym","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vulnlmp-openai","name":"VulnLMP","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from July 2026, for GPT-5.6 Sol."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"vwe-bench","name":"VWE-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","6 models we track, plus 9 other entries on its leaderboard."],["scores.recent","pass","The newest is from August 2026, for Qwen 3.8 Max."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 57.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on usail-hkust.github.io: 6 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wahl-o-mat-european-election-2024","name":"Wahl-O-Mat 2024 European election party agreement","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wahl-o-mat-bundestag-2025","name":"Wahl-O-Mat 2025 Bundestag party agreement","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wandr","name":"WANDR","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries), launch posts (1 score) and system cards (4 scores)."],["scores.models","pass","4 models we track, plus 6 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 82.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wanlp-2022-arabic-propaganda","name":"WANLP 2022 Shared Task on Propaganda Detection in Arabic","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","warn","The record links a board on docs.google.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"warc-bench","name":"WARC-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","9 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","pass","The best is 64.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 10 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"washington-post-midterm-voter-guides-2026","name":"Washington Post midterm voter guides","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on datawrapper.dwcdn.net: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"washington-post-political-lean","name":"Washington Post political lean evaluation","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Claude Opus 4.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 76.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on washingtonpost.com: 30 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wassce-benchmark-for-open-language-models","name":"WASSCE Benchmark for Open Language Models","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"waxal-asr-test","name":"WAXAL Speech Recognition Test","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wayfinder-code-search","name":"Wayfinder Code Search Benchmark","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wbench-meituan","name":"WBench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (44 entries)."],["scores.models","pass","6 models we track, plus 38 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Cosmos3-Nano."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on meituan-longcat.github.io: 6 rows for models we track and 38 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"we-math","name":"We-Math","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (33 entries), launch posts (10 scores) and system cards (9 scores)."],["scores.models","pass","34 models we track, plus 12 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Keye-VL-2.0-30B-A3B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen 3.6 Plus (#75)."],["scores.headroom","info","Saturated. The best is 89%, past 85% since February 2026."],["import.importer","pass","We read the board on we-math.github.io: 21 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wearableqa","name":"WearableQA","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (14 entries)."],["scores.models","pass","12 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Gemma 4 26B A4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 72.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 24 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"weatherqa-severe-weather","name":"WeatherQA (severe weather)","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (22 entries)."],["scores.models","pass","10 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 11 months ago, in December 2025, for GPT-5.2."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 62.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 15 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"weavebench","name":"WeaveBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","10 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 41.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 16 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"web-clone-bench","name":"Web Clone Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","6 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 62%, under 85%, so it still separates models."],["import.importer","pass","We read the board on plato.so: 40 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"nimble-web-context","name":"Web Context (Nimble)","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from October 2026, for MiniMax-M3.1-Flash."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 80%, under 85%, so it still separates models."],["import.importer","pass","We read the board on fireworks.ai: 12 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"web-ctf-anthropic","name":"Web CTF challenges (Anthropic)","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (6 scores)."],["scores.models","pass","6 models we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Claude Sonnet 4.6. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since February 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"web-of-lies-google","name":"Web of lies","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","The newest came 2 years ago, in July 2024, for Gemma 2 27B. Only the lab that runs it can add scores."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"web-ui-bench","name":"Web UI Bench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on webuibench.dev that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webarena","name":"WebArena","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (47 entries), launch posts (4 scores) and system cards (5 scores)."],["scores.models","pass","12 models we track, plus 42 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt 74B-A9B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","pass","The best is 74.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on webarena.dev: 6 rows for models we track and 42 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webarena-verified","name":"WebArena-Verified","summary":{"pass":6,"warn":0,"fail":0,"info":4,"text":"All 6 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (5 scores) and system cards (2 scores)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from October 2026, for LFM3-1.7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 69%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webbench-halluminate","name":"WebBench (Halluminate)","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for OpenAI Computer-Using Agent (CUA)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 76.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on webbench.ai: 2 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webcanvas","name":"WebCanvas","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for GPT-4o mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 28.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 8 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webdev-arena","name":"WebDev Arena","summary":{"pass":6,"warn":2,"fail":0,"info":2,"text":"6 of 8 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from boards Epoch AI copied (105 models) and launch posts (2 scores)."],["scores.models","pass","107 models we track."],["scores.recent","pass","The newest is from October 2026, for Gemini 4 Carbon."],["scores.frontier","pass","7 of the top 10 have a score, led by Gemini 4 Carbon (#3)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","warn","The record links a board on arena.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webexploitbench","name":"WebExploitBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 51.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 18 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webguard","name":"WebGuard","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webharbor","name":"WebHarbor","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 86.7%, past 85% since September 2026."],["import.importer","pass","We read the board on aiming-lab.github.io: 8 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"weblinx","name":"WebLINX","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (20 entries)."],["scores.models","pass","3 models we track, plus 17 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in November 2023, for GPT-4V."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 2-13B (#223)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on mcgill-nlp.github.io: 3 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webquest","name":"WebQuest","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for GLM-4.6V."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 69.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webquestions","name":"WebQuestions","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 3 years ago, in May 2023, for PaLM 2-L. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is PaLM 2-L (#198)."],["scores.headroom","pass","The best is 28.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webshop","name":"WebShop","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"websitebench","name":"WebsiteBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 26.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on website-bench.com: 8 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"websrc","name":"WebSRC","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on x-lance.github.io: 7 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webtailbench","name":"WebTailBench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","8 models we track."],["scores.recent","pass","The newest is from July 2026, for Fara1.5-4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 57.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 2 pairs differs by more than 10 points, the largest for Fara-7B (−10 pts)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webtailbench-refusals","name":"WebTailBench-Refusals","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries) and system cards (2 scores)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from June 2026, for Fara1.5-27B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 100%, past 85% since June 2026."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webvoyager","name":"WebVoyager","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries), launch posts (12 scores) and system cards (8 scores)."],["scores.models","pass","13 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Super VL."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Saturated. The best is 89.3%, past 85% since January 2025."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webvr","name":"WebVR","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (19 entries)."],["scores.models","pass","19 models we track."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Gemini 3.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on webvr-benchmark.github.io: 19 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"webwalkerqa","name":"WebWalkerQA","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (15 entries)."],["scores.models","pass","5 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in September 2024, for Qwen2.5-14B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o1-preview (#149)."],["scores.headroom","pass","The best is 33.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on tongyi-agent.github.io: 13 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wechat-pay-physical-client-tencent","name":"WeChat Pay Physical Client Evaluation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wegeneditbench","name":"WeGenEditBench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","4 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"weirdchat","name":"WeirdChat","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"weirdml","name":"WeirdML","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (131 entries)."],["scores.models","pass","115 models we track, plus 16 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","2 of the top 10 have a score, which is fine for a retired test. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 93.6%, past 85% since June 2026."],["import.importer","pass","We read the board on htihle.github.io: 146 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"weirdml-v3","name":"WeirdML v3","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","27 models we track."],["scores.recent","pass","The newest is from November 2026, for Claude Fable 5.2."],["scores.frontier","pass","8 of the top 10 have a score, led by Claude Fable 5.2 (#2)."],["scores.headroom","pass","The best is 43.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on htihle.github.io: 29 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wellnessbench-micro1","name":"WellnessBench","summary":{"pass":9,"warn":2,"fail":0,"info":1,"text":"9 of 11 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 66%, under 85%, so it still separates models."],["import.importer","pass","We read the board on micro1.ai: 12 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wenetspeech","name":"WenetSpeech","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (7 scores)."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wenetspeech-test-meeting","name":"WenetSpeech Test_Meeting","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wenetspeech-test-net","name":"WenetSpeech Test_Net","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on github.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wenetspeech-chuan-eval","name":"WenetSpeech-Chuan-Eval","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on arxiv.org: 13 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"millionaire-bench-de","name":"Wer wird Millionär? LLM Benchmark","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (96 entries)."],["scores.models","pass","80 models we track, plus 16 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for DeepSeek V4.1 Flash."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on koray.at: 84 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"weref-bench-tencent","name":"WeRef-Bench","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","2 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen3.5-9B (#128)."],["scores.headroom","pass","The best is 50.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"werewolf-arena","name":"Werewolf Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in December 2023, for Gemini 1.0 Pro."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wham","name":"WHAM!","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"whamr","name":"WHAMR!","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"when2call","name":"When2Call","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (24 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","11 models we track, plus 14 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen2.5-72B (#166)."],["scores.headroom","pass","The best is 64.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 10 rows for models we track and 14 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"where-did-my-data-go","name":"Where Did My Data Go?","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on kaggle.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"where2place","name":"Where2Place","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries) and system cards (4 scores)."],["scores.models","pass","6 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Cosmos3-Edge."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 64%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 3 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"skills-il-israel-knowledge","name":"Which AI Model Knows Israel Best?","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Mistral Large 4."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Saturated. The best is 99.3%, past 85% since April 2025."],["import.importer","pass","We read the board on agentskills.co.il: 39 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"white-box-model-internals-analysis-anthropic","name":"White-box analyses of model internals (Anthropic)","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"who-are-you-llm-self-identity","name":"Who Are You? LLM Self-Identity","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","12 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since August 2025."],["import.importer","pass","We read the board on kaggle.com: 12 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"whu-cdc","name":"WHU-CDC","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wic","name":"WiC","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries) and launch posts (1 score)."],["scores.models","pass","1 model we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in May 2023, for PaLM 2-L."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is PaLM 2-L (#198)."],["scores.headroom","pass","The best is 66.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on pilehvar.github.io: 15 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"widesearch","name":"WideSearch","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries), launch posts (17 scores) and system cards (6 scores)."],["scores.models","pass","25 models we track."],["scores.recent","pass","The newest is from October 2026, for Ling-3.1-flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 83.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on widesearch-seed.github.io: 17 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"widget-captioning","name":"Widget Captioning","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wiki-live-challenge","name":"Wiki Live Challenge","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","3 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 15 months ago, in August 2025, for GPT-5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 (#71)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on agentresearchlab.com: 3 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wiki-neutrality-corpus","name":"Wiki Neutrality Corpus","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wikibias","name":"WIKIBIAS","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wikicompany2industry-clustering","name":"WikiCompany2IndustryClustering","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wikifact-helm","name":"WikiFact (HELM)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wikilingua","name":"WikiLingua","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wikineural","name":"WikiNEuRal","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wikispeedia","name":"Wikispeedia","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wikisql","name":"WikiSQL","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (39 entries)."],["scores.models","pass","39 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 39 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wikitablequestions","name":"WikiTableQuestions","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on ppasupat.github.io: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wikitext-2","name":"WikiText-2","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","7 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 7 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wikivqabench-ibm","name":"WikiVQABench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wild-omnidocbench","name":"Wild-OmniDocBench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from October 2026, for MiniCPM-OCR-1.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 92.4%, past 85% since October 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildartifactbench-meta","name":"WildArtifactBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildawm-validation","name":"WildAWM validation","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","1 model we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Qwen3.5-4B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 22.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildbench","name":"WildBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (63 entries), launch posts (4 scores) and system cards (1 score)."],["scores.models","pass","50 models we track, plus 16 other entries on its leaderboard."],["scores.recent","warn","The newest came 10 months ago, in January 2026, for STEP3-VL-10B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (#158)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 47 rows for models we track and 16 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildchat","name":"WildChat (toxic conversations)","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries) and system cards (4 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 20 months ago, in March 2025, for GPT-4o. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o1 (#118)."],["scores.headroom","info","Saturated. The best is 98%, past 85% since September 2024."],["import.importer","pass","We read the board on cdn.openai.com: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildchat-non-toxic-refusals-anthropic","name":"WildChat non-toxic incorrect refusals (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 2 years ago, in October 2024, for Claude 3.5 Haiku. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude 3.5 Sonnet (October 2024) (#150)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildchat-toxic-refusals-anthropic","name":"WildChat toxic refusals (Anthropic)","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 2 years ago, in October 2024, for Claude 3.5 Haiku. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Claude 3.5 Sonnet (October 2024) (#150)."],["scores.headroom","info","Saturated. The best is 96.4%, past 85% since June 2024."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildclawbench","name":"WildClawBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries), launch posts (4 scores) and system cards (3 scores)."],["scores.models","pass","36 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Glimmer."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 71%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 45 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildgraphbench","name":"WildGraphBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 8 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildguardtest","name":"WildGuardTest","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (5 scores)."],["scores.models","pass","5 models we track."],["scores.recent","info","The newest came 11 months ago, in December 2025, for Olmo 3.1 32B Think. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 96.9%, past 85% since December 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildjailbreak","name":"WildJailbreak","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","info","Only 2 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 10 months ago, in January 2026, for K-EXAONE. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 98.5%, past 85% since December 2025."],["import.importer","pass","We read the board on arxiv.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildkhmerst","name":"WildKhmerST","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildlifebench-wildlife-knowledge","name":"WildlifeBench Wildlife Knowledge V1","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","7 models we track."],["scores.recent","pass","The newest is from October 2026, for Claude Haiku 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6.1 Sol (#6)."],["scores.headroom","pass","The best is 84.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on wildlifebench.org: 7 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildscore","name":"WildScore","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","6 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for GPT-5.4 Mini."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 75%, under 85%, so it still separates models."],["import.importer","pass","We read the board on MuTheoryEval: 7 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildspeech-bench","name":"WildSpeech-Bench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries), launch posts (4 scores) and system cards (2 scores)."],["scores.models","pass","5 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildvision-bench","name":"WildVision","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (22 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","14 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Nemotron Nano 12B v2 VL."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on raw.githubusercontent.com: 14 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wildvision-arena","name":"WildVision Arena","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (22 entries)."],["scores.models","pass","10 models we track, plus 12 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in April 2024, for Idefics2-8B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3 Opus (#173)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 10 rows for models we track and 12 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"windows-agent-arena","name":"Windows Agent Arena","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","5 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Qwen3-VL-235B-A22B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 50.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on microsoft.github.io: 4 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"windsurf-arena-mode","name":"Windsurf Arena Mode Leaderboard","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","9 models we track, plus 6 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Kimi K2.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 4.6 (#39)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on devin.ai: 12 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"winobias","name":"WinoBias","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Gemma 2 27B."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"winogender","name":"Winogender","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 2 years ago, in July 2024, for Gemma 2 27B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 2 27B (#190)."],["scores.headroom","pass","The best is 79.2%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"winograd","name":"Winograd","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 3 years ago, in May 2023, for PaLM 2-L. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is PaLM 2-L (#198)."],["scores.headroom","info","Saturated. The best is 89.5%, past 85% since May 2023."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"winogrande","name":"WinoGrande","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries), launch posts (21 scores), system cards (41 scores) and claims Epoch AI collected (76 models)."],["scores.models","pass","102 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Lightning 30B-A3B Base."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is DeepSeek-V4-Pro (#70)."],["scores.headroom","info","Saturated. The best is 91.7%, past 85% since April 2022."],["import.importer","pass","We read the board on arxiv.org: 8 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"winoground","name":"Winoground","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (20 entries)."],["scores.models","pass","20 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 20 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wise-t2i","name":"WISE","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries) and system cards (1 score)."],["scores.models","pass","12 models we track, plus 15 other entries on its leaderboard."],["scores.recent","info","The newest came 18 months ago, in May 2025, for BAGEL-7B-MoT. That’s expected for a test replaced by WISE Verified."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 12 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wise-verified","name":"WISE Verified","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries)."],["scores.models","pass","11 models we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 22 months ago, in January 2025, for Janus-Pro-7B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 12 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wiser-atomic","name":"WISER-Atomic","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","6 models we track, plus 4 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Gemini 2.5 Flash. That’s expected for a test replaced by WISER-Search."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is o3 (#84)."],["scores.headroom","pass","The best is 69%, under 85%, so it still separates models."],["import.importer","pass","We read the board on parallel.ai: 7 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wiser-findall","name":"WISER-FindAll","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","1 model we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 21 months ago, in February 2025, for OpenAI deep research."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 21%, under 85%, so it still separates models."],["import.importer","pass","We read the board on parallel.ai: 2 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wiser-search","name":"WISER-Search","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","12 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on parallel.ai: 12 entries at the last read, none a model we track."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wisesight-sentiment","name":"Wisesight Sentiment","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on github.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"witnessbench","name":"WitnessBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 models we track."],["scores.recent","pass","The newest is from September 2026, for Gemini 4 Argon."],["scores.frontier","pass","3 of the top 10 have a score, led by Claude Opus 5.5 (#5)."],["scores.headroom","pass","The best is 46.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on witnessbench.ai: 13 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wixqa","name":"WixQA","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmarena-video-arena","name":"WMArena Video Arena","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","4 models we track, plus 6 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on wmarena.com: 4 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmdp","name":"WMDP","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 84.8%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmdp-biology","name":"WMDP Biology","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries), launch posts (1 score) and system cards (8 scores)."],["scores.models","pass","10 models we track, plus 8 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Grok 4.20. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Saturated. The best is 91%, past 85% since February 2025."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmdp-chemistry","name":"WMDP Chemistry","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries), launch posts (1 score) and system cards (7 scores)."],["scores.models","pass","9 models we track, plus 8 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Beam 1.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Saturated. The best is 90%, past 85% since March 2026."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmdp-cyber","name":"WMDP Cyber","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries), launch posts (1 score) and system cards (2 scores)."],["scores.models","pass","4 models we track, plus 8 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Grok 4.20. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Grok 4.20 (#61)."],["scores.headroom","info","Saturated. The best is 91%, past 85% since March 2026."],["import.importer","pass","We read the board on wmdp.ai: 2 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt-2014-news-translation","name":"WMT 2014 News Translation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt16-news-translation","name":"WMT 2016 News Translation","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt-discourse-level-literary-translation-2023","name":"WMT 2023 Discourse-Level Literary Translation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries)."],["scores.models","pass","1 model we track, plus 9 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in June 2023, for GPT-4. That’s expected for a test replaced by WMT 2024 Discourse-Level Literary Translation."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-4 (#186)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 1 row for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt-discourse-level-literary-translation-2024","name":"WMT 2024 Discourse-Level Literary Translation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt23","name":"WMT23","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 2 years ago, in May 2024, for Gemini 1.5 Pro. That’s expected for a test replaced by WMT26."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 1.5 Pro (#174)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt24","name":"WMT24","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt24-icelandic-idiomatic-expressions","name":"WMT24 Icelandic Idiomatic Expressions","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (17 entries)."],["scores.models","pass","2 models we track, plus 15 other entries on its leaderboard."],["scores.recent","warn","The newest came 2 years ago, in August 2024, for Command R+."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3-70B (#182)."],["scores.headroom","pass","The best is 41.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 2 rows for models we track and 15 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt24-icelandic-proper-names","name":"WMT24 Icelandic Proper Names","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt24-plus-plus","name":"WMT24++","summary":{"pass":5,"warn":0,"fail":0,"info":5,"text":"All 5 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (25 scores) and system cards (18 scores)."],["scores.models","pass","15 models we track."],["scores.recent","pass","The newest is from October 2026, for Nemotron 3.5 Bolt E4B."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Qwen3.7-Max (#49)."],["scores.headroom","info","Saturated. The best is 90.5%, past 85% since September 2025."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt25","name":"WMT25","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from September 2026, for North Small Translate 1.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt25-eea-terminology-icelandic","name":"WMT25 EEA Terminology","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (34 entries)."],["scores.models","pass","13 models we track, plus 21 other entries on its leaderboard."],["scores.recent","warn","The newest came 17 months ago, in June 2025, for Gemini 2.5 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","pass","The best is 46.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 13 rows for models we track and 21 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt-mist-2025","name":"WMT25 Multilingual Instruction Shared Task","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","warn","The record links a board on github.com that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt25-terminology","name":"WMT25 Terminology Translation Task","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for North Small Translate 1.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 89.7%, past 85% since September 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt26","name":"WMT26","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from September 2026, for North Small Translate 1.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt-mist-2026","name":"WMT26 Multilingual Instruction Shared Task","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wmt26-video-subtitle-translation","name":"WMT26 Video Subtitle Translation","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","4 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on aclanthology.org: 1 row for models we track and 3 other entries at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wojak-bench-3d","name":"Wojak-Bench 3D","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","10 models we track."],["scores.recent","pass","The newest is from November 2026, for Gemini 4 Neon."],["scores.frontier","pass","7 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as an Elo rating, with no fixed ceiling."],["import.importer","pass","We read the board on wojak-bench.dev.pion.dev: 10 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wolf","name":"WOLF","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wl-bisindo","name":"Word-Level BISINDO","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"word-task-tencent","name":"Word-Task (Tencent)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 17 months ago, in June 2025, for Hunyuan-A13B-Instruct. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 62.9%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"word-edit-bench-tencent","name":"WordEditBench","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (7 entries)."],["scores.models","pass","6 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Tencent Hy4."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is DeepSeek V4.1 Flash 1017 (#23)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 6 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wordlearenas","name":"WordleArenas","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (45 entries)."],["scores.models","pass","32 models we track, plus 13 other entries on its leaderboard."],["scores.recent","info","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since May 2025."],["import.importer","pass","We read the board on drchangliu.github.io: 38 rows for models we track and 13 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"work360","name":"Work360","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","pass","The newest is from November 2026, for Qwen4-Max."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"workarena","name":"WorkArena","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"workarena-plus-plus","name":"WorkArena++","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","warn","The newest came 2 years ago, in May 2024, for GPT-4o."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o (#162)."],["scores.headroom","pass","The best is 1.90%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"workbuddy-user-blind-evaluation-tencent","name":"WorkBuddy User Blind Evaluation","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (4 entries)."],["scores.models","info","Only 3 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for DeepSeek V4.1 Flash 1017."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Haiku 5.5 (#20)."],["scores.headroom","info","Scored as a rating, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 3 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"workbuddy-bench-office-tencent","name":"WorkBuddyBench-Office","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","14 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","1 of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5.5 (#5)."],["scores.headroom","info","Saturated. The best is 86.4%, past 85% since September 2026."],["import.importer","pass","We read the board on workbuddybench.com: 19 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"workflowarena","name":"WorkflowArena","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","5 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 37%, under 85%, so it still separates models."],["import.importer","pass","We read the board on lab.einsia.ai: 25 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"workindex-tencent","name":"WorkIndex","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"snorkel-workplaceagents","name":"WorkplaceAgents","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","14 models we track."],["scores.recent","pass","The newest is from October 2026, for Muse Wonder."],["scores.frontier","pass","5 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 21.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on snorkel.ai: 14 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"workrb-skill-extraction-challenge-2026","name":"WorkRB Skill Extraction Challenge 2026","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (18 entries)."],["scores.models","pass","18 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on workrb.techwolf.ai: 18 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"workspacebench-lite","name":"Workspace-Bench-Lite","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","9 models we track, plus 1 other entry on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Grok 4.3."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 66.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on workspace-bench.github.io: 41 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"workspacebench","name":"WorkSpaceBench","summary":{"pass":4,"warn":2,"fail":0,"info":4,"text":"4 of 6 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (3 scores) and system cards (1 score)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from August 2026, for Tencent Hy4 preview."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen 3.8 Max (#30)."],["scores.headroom","pass","The best is 67.7%, under 85%, so it still separates models."],["import.importer","warn","The record links a board on workspace-bench.github.io that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worksurface-bench","name":"WorkSurface-Bench","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","4 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 4 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"world-class-bio-securebio","name":"World-Class Bio","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (86 entries) and system cards (5 scores)."],["scores.models","pass","79 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 68.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on securebio.org: 79 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 4 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"world-in-world","name":"World-in-World","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries)."],["scores.models","pass","2 models we track, plus 9 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on world-in-world.github.io: 2 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldarena","name":"WorldArena","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (125 entries)."],["scores.models","pass","3 models we track, plus 122 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 3 rows for models we track and 122 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldarena-2-0","name":"WorldArena 2.0","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (81 entries)."],["scores.models","pass","1 model we track, plus 80 other entries on its leaderboard."],["scores.recent","warn","The newest came 14 months ago, in September 2025, for Code World Model (CWM)."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 69.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on huggingface.co: 1 row for models we track and 80 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldauditbench","name":"WorldAuditBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6 Astra."],["scores.frontier","info","Only 1 of the top 10 has a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 42.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ucsb-nlp-chang.github.io: 10 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldbench","name":"WorldBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries) and system cards (1 score)."],["scores.models","pass","13 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 67.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on worldbench-vl.github.io: 14 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldcoder-bench","name":"WorldCoder-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 27.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldexam","name":"WorldExam","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","4 models we track, plus 5 other entries on its leaderboard."],["scores.recent","info","No scores yet."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on worldexam.github.io: 4 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldgui","name":"WorldGUI","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","1 model we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for GPT-5.1."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.1 (#66)."],["scores.headroom","pass","The best is 45.8%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 1 row for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldmodelbench","name":"WorldModelBench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (13 entries)."],["scores.models","pass","13 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on worldmodelbench-team.github.io: 13 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldmodelgym","name":"WorldModelGym","summary":{"pass":2,"warn":3,"fail":0,"info":5,"text":"2 of 5 checks pass, 3 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","warn","The record links a board on wm-gym.labs.reka.ai that we don’t read yet."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldolympiad","name":"WorldOlympiad","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (8 entries)."],["scores.models","pass","8 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on alibaba-damo-academy.github.io: 8 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldprediction","name":"WorldPrediction","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldscore","name":"WorldScore","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (35 entries)."],["scores.models","pass","35 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on huggingface.co: 35 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldsense","name":"WorldSense","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries), launch posts (1 score) and system cards (3 scores)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for Seed2.0 Lite (0428)."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","pass","The best is 67.3%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldsense-omnimodal","name":"WorldSense (omnimodal)","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries), launch posts (6 scores) and system cards (5 scores)."],["scores.models","pass","15 models we track, plus 17 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Qwen3.8-Omni-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 68.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on jaaackhongggg.github.io: 12 rows for models we track and 17 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","All 3 pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldsimbench","name":"WorldSimBench","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldstate-bench","name":"WorldState-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (12 entries)."],["scores.models","pass","3 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Wan2.2-I2V-A14B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 32%, under 85%, so it still separates models."],["import.importer","pass","We read the board on ivg-yanranzhang.github.io: 4 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldtravel","name":"WorldTravel","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries) and launch posts (1 score)."],["scores.models","pass","8 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Seed1.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","pass","The best is 47.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 12 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"worldvqa","name":"WorldVQA","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries), launch posts (8 scores) and system cards (4 scores)."],["scores.models","pass","31 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Muse Spark Open."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Kimi K3 (#25)."],["scores.headroom","pass","The best is 61.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on worldvqa2026.github.io: 23 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wrench-compose","name":"WRENCH (Docker Compose)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries)."],["scores.models","info","Only 3 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 5 (#35)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 3 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wrench-factorio","name":"WRENCH (Factorio)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Sonnet 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 5 (#35)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 2 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"writing-benchmark-every","name":"Writing Benchmark (Every)","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from June 2026, for Claude Fable 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on every.to: 4 rows for models we track at the last read."],["import.read","pass","November 21st, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"writing-style-matching-apaz","name":"Writing Style Matching (apaz)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"writingbench","name":"WritingBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (49 entries), launch posts (10 scores) and system cards (11 scores)."],["scores.models","pass","57 models we track, plus 4 other entries on its leaderboard."],["scores.recent","pass","The newest is from July 2026, for LongCat-2.0."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on huggingface.co: 50 rows for models we track and 4 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wsc","name":"WSC","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries) and launch posts (1 score)."],["scores.models","pass","4 models we track, plus 23 other entries on its leaderboard."],["scores.recent","info","The newest came 3 years ago, in May 2023, for PaLM 2-L. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is PaLM 2-L (#198)."],["scores.headroom","info","Saturated. The best is 95.9%, past 85% since October 2019."],["import.importer","pass","We read the board on super.gluebenchmark.com: 3 rows for models we track and 23 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wsi-path-google","name":"WSI-Path","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wsyue-eval","name":"WSYue-eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"wtp-bench","name":"WTP-Bench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","24 models we track, plus 3 other entries on its leaderboard."],["scores.recent","warn","The newest came 13 months ago, in October 2025, for Claude Haiku 4.5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Sonnet 4.5 (#81)."],["scores.headroom","pass","The best is 79%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sochastic.me: 24 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"x-browse-xai","name":"X Browse (xAI)","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries) and launch posts (2 scores)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Grok 4.1 Fast."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","pass","The best is 58%, under 85%, so it still separates models."],["import.importer","pass","We read the board on x.ai: 4 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"x-stance","name":"x-stance","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on ceur-ws.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"x-stream","name":"X-Stream","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (12 entries)."],["scores.models","pass","5 models we track, plus 7 other entries on its leaderboard."],["scores.recent","warn","The newest came 12 months ago, in November 2025, for Gemini 3 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3 Pro (#56)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track and 7 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xacle-2026","name":"XACLE Challenge 2026","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (49 entries)."],["scores.models","pass","49 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on xacle.org: 49 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xai-factuality-hallucination-eval","name":"xAI Factuality (Hallucination) eval","summary":{"pass":7,"warn":1,"fail":0,"info":4,"text":"7 of 8 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the official leaderboard (2 entries) and system cards (2 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from August 2026, for Grok 4.6."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on x.ai: 2 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xai-refusal-and-jailbreak-suites","name":"xAI refusal and jailbreak suites","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xai-sycophancy-eval","name":"xAI sycophancy eval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (1 score) and system cards (4 scores)."],["scores.models","info","Only 4 models we track."],["scores.recent","pass","The newest is from September 2026, for Grok 4.7."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is Grok 4.6 (#29)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xbench","name":"Xbench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","17 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","pass","3 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on offpolicyinc.com: 17 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xbench-deepsearch","name":"xbench-DeepSearch","summary":{"pass":9,"warn":0,"fail":0,"info":3,"text":"All 9 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries), launch posts (3 scores) and system cards (1 score)."],["scores.models","pass","4 models we track, plus 10 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for Step 3.5 Flash. That’s expected for a test replaced by xbench-DeepSearch-2510."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K2.5 (#76)."],["scores.headroom","pass","The best is 83.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on admin.xbench-ai.com: 1 row for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xbench-deepsearch-2510","name":"xbench-DeepSearch-2510","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (11 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","3 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 9 months ago, in February 2026, for Step 3.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5 Pro (#64)."],["scores.headroom","pass","The best is 79%, under 85%, so it still separates models."],["import.importer","pass","We read the board on xbench.org: 2 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xbench-profession-marketing","name":"xbench-Marketing","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","7 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on xbench.org: 7 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xbench-profession-recruitment","name":"xbench-Recruitment","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (10 entries)."],["scores.models","pass","8 models we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is o3 (#84)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on xbench.org: 8 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xbench-scienceqa","name":"xbench-ScienceQA","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (29 entries)."],["scores.models","pass","29 models we track."],["scores.recent","pass","The newest is from September 2026, for Claude Opus 5.5."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 76.4%, under 85%, so it still separates models."],["import.importer","pass","We read the board on xbench.org: 30 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xbow-validation-benchmarks","name":"XBOW Validation Benchmarks","summary":{"pass":2,"warn":1,"fail":0,"info":7,"text":"2 of 3 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Claude Opus 4.7. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 4.7 (#31)."],["scores.headroom","info","Saturated. The best is 98.5%, past 85% since April 2026."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xchatbench","name":"xChatBench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Aya Vision 32B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xcodeeval","name":"xCodeEval","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","Only 2 models we track."],["scores.recent","warn","The newest came 2 years ago, in February 2024, for StarCoder 2 3B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is StarCoder 2 3B (#252)."],["scores.headroom","pass","The best is 3.81%, under 85%, so it still separates models."],["import.importer","pass","We read the board on aclanthology.org: 3 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xcopa","name":"XCOPA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (3 entries) and launch posts (2 scores)."],["scores.models","pass","2 models we track, plus 3 other entries on its leaderboard."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Apertus 70B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is PaLM 2-L (#198)."],["scores.headroom","info","Saturated. The best is 94.4%, past 85% since May 2023."],["import.importer","pass","We read the board on github.com: 3 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xdailybench-bytedance","name":"xDailyBench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (1 score) and system cards (1 score)."],["scores.models","pass","4 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from June 2026, for Seed2.1 Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 73%, under 85%, so it still separates models."],["import.importer","pass","We read the board on seed.bytedance.com: 4 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xdev-bench-hard-moonshot","name":"xdev-bench-hard","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (6 entries) and launch posts (1 score)."],["scores.models","pass","5 models we track, plus 1 other entry on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Kimi K3.1 Pro."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.6 Sol (#14)."],["scores.headroom","info","Saturated. The best is 100%, past 85% since July 2026."],["import.importer","pass","We read the board on kimi.com: 5 rows for models we track and 1 other entry at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xl-sum-english","name":"XL-Sum (English)","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xlrs-bench","name":"XLRS-Bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (10 entries), launch posts (3 scores) and system cards (3 scores)."],["scores.models","pass","11 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Intern-S2-397B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-4o mini (#178)."],["scores.headroom","pass","The best is 55%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 14 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xmmmu","name":"xMMMU","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries) and system cards (1 score)."],["scores.models","pass","6 models we track, plus 9 other entries on its leaderboard."],["scores.recent","warn","The newest came 18 months ago, in May 2025, for Aya Vision 32B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 45.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on neulab.github.io: 5 rows for models we track and 9 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xnli","name":"XNLI","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 14 months ago, in September 2025, for Apertus 70B. Labs stopped reporting it, and no one else runs it."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 45.3%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xnlieu","name":"XNLIeu","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (9 entries)."],["scores.models","pass","9 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 9 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xor-tydi-qa","name":"XOR-TyDi QA","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries)."],["scores.models","pass","15 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on nlp.cs.washington.edu: 15 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xpertbench-bytedance","name":"XpertBench","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from launch posts (2 scores) and system cards (1 score)."],["scores.models","info","Only 3 models we track."],["scores.recent","info","The newest came 7 months ago, in April 2026, for Seed2.0 Lite (0428). Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 64.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xquad","name":"XQuAD","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (4 entries), launch posts (1 score) and system cards (4 scores)."],["scores.models","pass","4 models we track, plus 4 other entries on its leaderboard."],["scores.recent","warn","The newest came 20 months ago, in March 2025, for Gemma 3 12B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemma 3 27B (#165)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 4 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xstest","name":"XSTest","summary":{"pass":4,"warn":1,"fail":0,"info":5,"text":"4 of 5 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from system cards (16 scores)."],["scores.models","pass","7 models we track."],["scores.recent","warn","The newest came 8 months ago, in March 2026, for Seed1.8."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude 3.5 Sonnet (October 2024) (#150)."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xstorycloze","name":"XStoryCloze","summary":{"pass":7,"warn":2,"fail":0,"info":3,"text":"7 of 9 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","Only 1 model we track, plus 2 other entries on its leaderboard."],["scores.recent","warn","The newest came 6 years ago, in June 2020, for GPT-3 6.7B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 58.2%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 2 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xsum","name":"XSum","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 5 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xtd-200","name":"XTD-200","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xtreme","name":"XTREME","summary":{"pass":7,"warn":0,"fail":1,"info":4,"text":"7 of 8 checks pass, 1 problem"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (27 entries)."],["scores.models","pass","27 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on sites.research.google: 27 entries at the last read, none a model we track."],["import.read","pass","November 18th, 2026."],["import.failed","fail","The read on November 18th, 2026 failed with “502 https://sites.research.google/xtreme/app.js”. Pages show the rows from November 18th, 2026."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xtreme-r","name":"XTREME-R","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on arxiv.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xwinograd","name":"XWinograd","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xybench","name":"XYBench","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Pro."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 3.1 Pro (#40)."],["scores.headroom","pass","The best is 31%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"xyeval-google","name":"XYEval","summary":{"pass":8,"warn":2,"fail":0,"info":2,"text":"8 of 10 checks pass, 2 warnings"},"checks":[["record.access","warn","We don’t know whether the tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from August 2026, for Gemini 3.7 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.5 (#21)."],["scores.headroom","pass","The best is 60.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 5 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"yc-bench","name":"YC-Bench","summary":{"pass":10,"warn":1,"fail":0,"info":1,"text":"10 of 11 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (69 entries) and launch posts (1 score)."],["scores.models","pass","66 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for GPT-6.2 Astra."],["scores.frontier","pass","10 of the top 10 have a score, led by GPT-6.2 Astra (#1)."],["scores.headroom","info","Scored as US dollars, with no fixed ceiling."],["import.importer","pass","We read the board on collinear-ai.github.io: 67 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","warn","1 of 1 pair differs by more than 10 points, the largest for MiniMax-M3 (+$1,100,494)."],["consistency.issues","pass","No published audit found problems."]]},{"id":"yakk","name":"Yet Another Kernel Kernel","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a speedup, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"yocausal","name":"YoCausal","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (14 entries)."],["scores.models","pass","4 models we track, plus 10 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Wan2.2-T2V-A14B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on youzhexie.me: 4 rows for models we track and 10 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"yorulect","name":"YorùLect","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (3 entries)."],["scores.models","info","3 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on aclanthology.org: 3 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"youcook2","name":"YouCook2","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries), launch posts (2 scores) and system cards (2 scores)."],["scores.models","pass","4 models we track, plus 5 other entries on its leaderboard."],["scores.recent","warn","The newest came 16 months ago, in July 2025, for Gemini 2.5 Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Gemini 2.5 Pro (#97)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"youcook2-dense-description","name":"YouCook2 Dense Video Description","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (1 entry)."],["scores.models","info","1 entry on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on github.com: 1 entry at the last read, none a model we track."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"youcook2-retrieval","name":"YouCook2 retrieval","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score) and system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 6 months ago, in May 2026, for Gemini Embedding 2."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"your-llm-doesn-t-know-boundary","name":"Your LLM doesn't Know BOUNDARY!.","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (9 entries)."],["scores.models","pass","9 models we track."],["scores.recent","pass","The newest is from September 2026, for GPT-6.1 Sol."],["scores.frontier","pass","4 of the top 10 have a score, led by GPT-6 Astra (#4)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on kaggle.com: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"your-voice-your-device-your-language","name":"Your Voice, Your Device, Your Language Challenge","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on zindi.africa: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"youtube-vis","name":"YouTube-VIS","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"youtube-vos","name":"YouTube-VOS","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","pass","We read the board on arxiv.org: 6 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"yupp-vibe-score","name":"Yupp VIBE Score","summary":{"pass":2,"warn":2,"fail":0,"info":6,"text":"2 of 4 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","Yupp blocks automated reading, so we link to its board instead."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"z-ai-code-bench","name":"Z.ai Code Bench","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries) and launch posts (1 score)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from August 2026, for GLM-5.3-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Fable 5 (#12)."],["scores.headroom","pass","The best is 39.5%, under 85%, so it still separates models."],["import.importer","pass","We read the board on z.ai: 6 rows for models we track at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","The one pair agrees within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"zalo-ai-challenge-2022-end-to-end-question-answering","name":"Zalo AI Challenge 2022 End-to-End Question Answering","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"zclawbench","name":"ZClawBench","summary":{"pass":3,"warn":2,"fail":0,"info":5,"text":"3 of 5 checks pass, 2 warnings"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","warn","Likely. Some facts rest on secondary sources."],["scores.sources","info","Only labs’ own numbers, so no one else has checked them. Scores from launch posts (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for GLM-5V-Turbo."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 57.6%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"zebralogic","name":"ZebraLogic","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (58 entries), launch posts (16 scores) and system cards (19 scores)."],["scores.models","pass","70 models we track, plus 11 other entries on its leaderboard."],["scores.recent","info","The newest came 9 months ago, in February 2026, for ERNIE 5.0. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Kimi K2 (#117)."],["scores.headroom","info","Saturated. The best is 97.3%, past 85% since January 2025."],["import.importer","pass","We read the board on huggingface.co: 54 rows for models we track and 11 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"zendoworld","name":"ZendoWorld","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"zenml-bench","name":"ZenML Bench","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"zerobench","name":"ZeroBench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (80 entries), launch posts (27 scores) and system cards (17 scores)."],["scores.models","pass","103 models we track, plus 3 other entries on its leaderboard."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Mini."],["scores.frontier","info","Only 2 of the top 10 have a score. The strongest tested is GPT-6 Astra (#4)."],["scores.headroom","pass","The best is 54.6%, under 85%, so it still separates models."],["import.importer","pass","We read the board on zerobench.github.io: 124 rows for models we track and 3 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 6 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]},{"id":"zeroclue","name":"ZeroCLUE","summary":{"pass":8,"warn":1,"fail":0,"info":3,"text":"8 of 9 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (450 entries)."],["scores.models","pass","1 model we track, plus 449 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in March 2023, for ChatGLM-6B."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on cluebenchmarks.com: 4 rows for models we track and 449 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"zerolang-editing","name":"Zerolang Editing","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"zeroscrolls-quality","name":"ZeroSCROLLS/QuALITY","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (8 entries), launch posts (1 score) and system cards (3 scores)."],["scores.models","pass","4 models we track, plus 7 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in November 2024, for Llama 3.1-405B. That’s expected for a saturated test."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is Llama 3.1-405B (#163)."],["scores.headroom","info","Saturated. The best is 95.2%, past 85% since July 2024."],["import.importer","pass","We read the board on arxiv.org: 1 row for models we track and 7 other entries at the last read."],["import.read","pass","November 19th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"zerovideo-bytedance","name":"ZeroVideo","summary":{"pass":4,"warn":0,"fail":0,"info":6,"text":"All 4 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (2 scores)."],["scores.models","info","Only 2 models we track."],["scores.recent","info","The newest came 8 months ago, in March 2026, for Seed1.8. Only the lab that runs it can add scores."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 14.5%, under 85%, so it still separates models."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"zhoblimp","name":"ZhoBLiMP","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (7 entries)."],["scores.models","pass","2 models we track, plus 5 other entries on its leaderboard."],["scores.recent","info","The newest came 2 years ago, in September 2024, for Qwen2.5-14B. That’s expected for a saturated test."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","info","Saturated. The best is 87.2%, past 85% since September 2024."],["import.importer","pass","We read the board on aclanthology.org: 6 rows for models we track and 5 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"ziqi-eval","name":"ZIQI-Eval","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (17 entries)."],["scores.models","pass","9 models we track, plus 8 other entries on its leaderboard."],["scores.recent","warn","The newest came 3 years ago, in September 2023, for Qwen-7B."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Qwen-14B (#214)."],["scores.headroom","pass","The best is 26.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on github.com: 18 rows for models we track and 8 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"zmultidialbench-zai","name":"ZMultiDialBench","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for GLM-5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GLM-5 (#89)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"zmultitransbench-zai","name":"ZMultiTransBench","summary":{"pass":3,"warn":0,"fail":0,"info":7,"text":"All 3 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","info","Only labs’ own numbers, as expected for an in-house eval. Scores from system cards (1 score)."],["scores.models","info","Only 1 model we track."],["scores.recent","info","The newest came 9 months ago, in February 2026, for GLM-5. Only the lab that runs it can add scores."],["scores.frontier","info","None of the top 10 has a score, which is fine for an in-house eval. The strongest tested is GLM-5 (#89)."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"zoomed-in-bvcc","name":"Zoomed-in BVCC","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"owe-voice","name":"Òwe-Voice","summary":{"pass":7,"warn":0,"fail":0,"info":5,"text":"All 7 checks pass"},"checks":[["record.access","pass","The tasks are private."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (2 entries)."],["scores.models","info","2 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Lower is better, so there’s no ceiling to reach."],["import.importer","pass","We read the board on aclanthology.org: 2 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"csfd-sentiment","name":"ČSFD Sentiment","summary":{"pass":3,"warn":1,"fail":0,"info":6,"text":"3 of 4 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","warn","No scores from any source yet."],["scores.models","info","No models scored yet."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","No scores yet."],["import.importer","info","No public leaderboard to read."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"poleval-smigiel","name":"ŚMIGIEL","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","Public tasks, plus a private test set."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 entries on its leaderboard, none a model we track."],["scores.recent","info","No scores yet."],["scores.frontier","info","No models scored yet."],["scores.headroom","info","Scored as a plain number, with no fixed ceiling."],["import.importer","pass","We read the board on 2025.poleval.pl: 5 entries at the last read, none a model we track."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau-bench-2-0","name":"τ-bench 2.0","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","pass","The newest is from September 2026, for Gemini 3.8 Live Extended Thinking."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 79.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sierra-tau-bench-public.s3.us-west-2.amazonaws.com: 3 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau-rec","name":"τ-Rec","summary":{"pass":9,"warn":1,"fail":0,"info":2,"text":"9 of 10 checks pass, 1 warning"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (6 entries)."],["scores.models","pass","6 models we track."],["scores.recent","warn","The newest came 7 months ago, in April 2026, for DeepSeek-V4-Flash."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is GPT-5.4 (#28)."],["scores.headroom","pass","The best is 57.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on arxiv.org: 9 rows for models we track at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau-voice","name":"τ-Voice","summary":{"pass":11,"warn":0,"fail":0,"info":1,"text":"All 11 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (15 entries) and launch posts (4 scores)."],["scores.models","pass","10 models we track, plus 7 other entries on its leaderboard."],["scores.recent","pass","The newest is from October 2026, for Grok Voice 3.0."],["scores.frontier","info","None of its models are on the capabilities index."],["scores.headroom","pass","The best is 81.7%, under 85%, so it still separates models."],["import.importer","pass","We read the board on taubench.com: 11 rows for models we track and 6 other entries at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","pass","Both pairs agree within 10 points."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau-tau-bench","name":"τ^τ-bench","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (5 entries)."],["scores.models","pass","5 models we track."],["scores.recent","pass","The newest is from July 2026, for Claude Opus 5."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 23.9%, under 85%, so it still separates models."],["import.importer","pass","We read the board on sierra-research.github.io: 6 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau-c-bench","name":"τc-Bench","summary":{"pass":8,"warn":0,"fail":0,"info":4,"text":"All 8 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the benchmark’s paper (5 entries)."],["scores.models","pass","3 models we track, plus 2 other entries on its leaderboard."],["scores.recent","info","The newest came 11 months ago, in December 2025, for Gemini 3 Flash. That’s expected for a test replaced by τ-bench 2.0."],["scores.frontier","info","None of the top 10 has a score, which is fine for a retired test. The strongest tested is GPT-5.2 (#51)."],["scores.headroom","info","Saturated. The best is 87.7%, past 85% since December 2025."],["import.importer","pass","We read the board on arxiv.org: 6 rows for models we track and 2 other entries at the last read."],["import.read","pass","November 20th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","No model has both a lab number and an independent run."],["consistency.issues","pass","No published audit found problems."]]},{"id":"tau3-banking","name":"τ³-bench Banking","summary":{"pass":10,"warn":0,"fail":0,"info":2,"text":"All 10 checks pass"},"checks":[["record.access","pass","All tasks are public."],["record.confidence","pass","Checked against a primary source."],["scores.sources","pass","Scores from the official leaderboard (26 entries), launch posts (20 scores) and system cards (11 scores)."],["scores.models","pass","45 models we track."],["scores.recent","pass","The newest is from November 2026, for Inkling 2 Small."],["scores.frontier","info","None of the top 10 has a score. The strongest tested is Claude Opus 5 (#11)."],["scores.headroom","pass","The best is 55.1%, under 85%, so it still separates models."],["import.importer","pass","We read the board on taubench.com: 27 rows for models we track at the last read."],["import.read","pass","November 18th, 2026."],["import.failed","pass","The last read worked."],["consistency.claims","info","1 large gap of 3 pairs, each explained in our notes."],["consistency.issues","pass","No published audit found problems."]]}]}