Benchmark Almanac

Law, finance and medicine

We track 194 benchmarks in law, finance and medicine. They run from everyday level to research frontier level.

29 in use 26 past 85% 85 with public code

Anthropic held the top law, finance and medicine score 37% of the contested time

That's about 17 of the 46 months since January 2023, averaged over these benchmarks. We only count time while other labs are still testing, and stop once a model passes 85%. Faded stretches no longer count.

Anthropic Google DeepMind Meta OpenAI xAI Other labs Qwen Moonshot AI
Everyday Most adults, without training
MortgageTax Not tested since Sep 2026
Claude Opus 5 72.1%
Expert Specialists with years of training, often over hours of work
MedQA Passed 85% in May 2024, by GPT-4o
GPT-4o 86.3%
LegalBench Passed 85% in Aug 2025, by GPT-5
Claude Fable 5 88.6%
TaxEval v2 Not tested since Sep 2026
Muse Spark 1.2 80.4%
CorpFin v2 Not tested since Aug 2026
Claude Opus 5 73.2%
Vals Finance Agent v2.1 18 leaders
Claude Fable 5.2 69.5%
CaseLaw v2 Not tested since Apr 2026
Grok 4.3 79.3%
MedCode 3 leaders
Claude Opus 5 63.6%
MedScribe Passed 85% in Feb 2026, by GPT-5.1
Claude Opus 5.5 91.4%
Harvey LAB v1.1 (HLAB-Vals-1.1) 7 leaders
Gemini 4 Argon 27.1%
Legal Research Bench 4 leaders
Gemini 4 Carbon 55.8%
Excel Modeling Benchmark v1.1 4 leaders
Claude Fable 5.2 77.4%
Tax Agent Bench 3 leaders
Gemini 4 Hydrogen 52.3%

Source: Official leaderboards

Each benchmark counts from the one source its own page leads with: its official leaderboard when that is current, else Epoch AI’s own runs, else labs’ own numbers. We date independent runs by the model’s release and lab reports by when they came out.

Anthropic Google DeepMind Meta OpenAI xAI Other labs Qwen Moonshot AI
Everyday Most adults, without training
MortgageTax Not tested since Sep 2026
Claude Opus 5 72.1%
Expert Specialists with years of training, often over hours of work
MedQA Passed 85% in May 2024, by GPT-4o
GPT-4o 86.3%
LegalBench Passed 85% in Aug 2025, by GPT-5
Claude Fable 5 88.6%
TaxEval v2 Not tested since Sep 2026
Muse Spark 1.2 80.4%
CorpFin v2 Not tested since Aug 2026
Claude Opus 5 73.2%
Vals Finance Agent v2.1 18 leaders
Claude Fable 5.2 69.5%
CaseLaw v2 Not tested since Apr 2026
Grok 4.3 79.3%
MedCode 3 leaders
Claude Opus 5 63.6%
MedScribe Passed 85% in Feb 2026, by GPT-5.1
Claude Opus 5.5 91.4%
Harvey LAB v1.1 (HLAB-Vals-1.1) 7 leaders
Gemini 4 Argon 27.1%
Legal Research Bench 4 leaders
Gemini 4 Carbon 55.8%
Excel Modeling Benchmark v1.1 4 leaders
Claude Fable 5.2 77.4%
Tax Agent Bench 3 leaders
Gemini 4 Hydrogen 52.3%

Source: Official leaderboards

Every independent run: each benchmark’s own leaderboard, Epoch AI’s runs and boards Epoch AI copied, dated by the model’s release. We leave out benchmarks that only labs report.

Anthropic Google DeepMind Meta OpenAI xAI Other labs Qwen Moonshot AI Thinking Machines
Everyday Most adults, without training
MortgageTax Not tested since Sep 2026
Claude Opus 5 72.1%
Expert Specialists with years of training, often over hours of work
MedQA Passed 85% in May 2024, by GPT-4o
GPT-4o 89.4%
LegalBench Passed 85% in Aug 2025, by GPT-5
Claude Fable 5 88.6%
TaxEval v2 Not tested since Sep 2026
Muse Spark 1.2 80.4%
CorpFin v2 Not tested since Aug 2026
Claude Opus 5 73.2%
Vals Finance Agent v2.1 18 leaders
Claude Fable 5.2 69.5%
CaseLaw v2 Not tested since Apr 2026
Grok 4.3 79.3%
MedCode 3 leaders
Claude Opus 5 63.6%
MedScribe Passed 85% in Feb 2026, by GPT-5.1
Claude Opus 5.5 91.4%
Harvey LAB v1.1 (HLAB-Vals-1.1) 7 leaders
Gemini 4 Argon 27.1%
Legal Research Bench 4 leaders
Gemini 4 Carbon 55.8%
Excel Modeling Benchmark v1.1 4 leaders
Claude Fable 5.2 77.4%
Tax Agent Bench 3 leaders
Inkling 2 Small 65.8%

Sources: Official leaderboards and labs’ model cards

Every score, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

From everyday to research frontier level

Grouped by who could solve the tasks, from things most adults can do to open research problems. The bar under each name is its best score.

Everyday

Most adults, without training

School

Students, from grade school to high school

University

Undergraduates and entry-level professionals

Expert

Specialists with years of training, often over hours of work

Research frontier

Researchers, on open problems or tasks that take days

By kind

Law

BenchmarkLevelScoredBest scoreStatus
LegalBench Models do 162 legal reasoning tasks written by legal professionals, covering six types of legal reasoning such as issue spotting and rule application. Expert Checked automatically 88.6% Not in cards
Legal Research Bench Agents answer US federal and state legal research questions with case-law and web tools. Expert rubrics, all-pass and weighted scores by LLM judge. Expert Graded by a model 55.8% In use
Harvey LAB (Legal Agent Benchmark) Agent works through legal matter files across 24+ practice areas and produces work product. Expert rubric, task passes only if every criterion passes. Expert Graded by a model 25.4% In use
CaseLaw v2 Models answer private questions over recent Canadian court decisions, such as finding key cases and answering across several documents. Expert Graded by a model 79.3% Not in cards
PRBench Model answers 1,100 expert-written law and finance questions. LLM judge grades against weighted expert rubrics of 10-30 criteria each. Expert Graded by a model 61.6% In use
Harvey LAB v1.1 (HLAB-Vals-1.1) Agents produce legal work product from matter files with file tools and docx, pptx and xlsx skills. A task passes only if every expert rubric criterion passes. Expert Graded by a model 27.1% In use
TRLawBench Models answer 297 questions on Turkish law in three stages: 97 questions from official ÖSYM legal exams and 200 harder questions across 14 branches of law. Answers are checked against the answer key. A judge model also grades the reasoning from 0 to 10. University Checked automatically 9.85 Not in cards
LEXam Models answer 7,537 questions from 340 law exams at Swiss, EU and international law courses, in English and German. Open answers are judged against reference answers, and multiple-choice items are exact match. Expert Graded by a model 70.2% Not in cards
prinzbench Models research fact patterns under U.S. statutes and regulations and hunt for obscure public information online. 33 private questions run three times give a score out of 99. Expert Graded by people 91.0 Not in cards
Immigration Bench Models answer expert-written US immigration law questions from seven personas, such as F-1 students, employers and DACA recipients. An AI grader checks each answer against an expert rubric. Expert Graded by a model 75.4% Not in cards
Legora BAR Models work private legal cases end to end inside Legora's agent harness, with the matter's documents and templates. An LLM judge scores outputs against rubrics written by lawyers. Expert Graded by a model 106.0 Not in cards
DAYJOB: Law Agents do legal assignments from a short request and a workspace of files. An agent judge applies an expert rubric, and a pass needs 95% of criteria. Expert Graded by a model 36.9% Not in cards
SnorkelLegal Agents work 200 tasks inside a simulated mid-market law firm. Each request ties to a matter, a procedural stage and a role with set authority. Expert Graded by a model 46% Not in cards
LEXam-hard Agents with legal research tools answer the 518 LEXam open questions that strong open models score lowest on. Questions answerable from memory or unanswerable from their text are removed. Research frontier Graded by a model 40.8% Not in cards
PLawBench Model handles 850 realistic legal consultation, case-analysis and document-drafting tasks. LLM judge scores against about 12,500 expert rubric items. Expert Graded by a model 66.5% In use
LegalReasonBench Models work long-horizon legal matters built by practicing lawyers, writing memoranda and court filings across dependent assignments, in closed-record and open-research settings. Expert Graded by a model 50.8% Not in cards
Ivo-micro1 ContractBench Models review contracts and return redlines across 350 attorney-written tasks in 35 scenarios and five agreement types. Tasks cover first review and replies to a counterparty. Expert Graded by a model 64% Not in cards
KBL Models answer Korean legal knowledge and reasoning tasks and Korean bar exam questions. It has 7 knowledge tasks, 4 reasoning tasks and 2,510 bar exam items. Expert Checked automatically 77.5% In use
BigLaw Bench Models review legal documents and receive rubric scores for substantive accuracy and editing quality. Research frontier Graded by a model 91% Fading
Harvey LAB-AA v1.1 Artificial Analysis's run of Harvey's Legal Agent Benchmark on Harvey's newer private set of 120 tasks across 24 practice areas. A deliverable with a material hallucination scores zero, and the headline is the share of judges who find every rubric criterion passed. Expert Graded by a model 13.1% In use
StrataBench Land v1 Extract Models extract ten shared lease fields from 628 synthetic instruments. Collide divides correct field decisions by correct, omitted, and wrong decisions, excluding unscored fields. Expert Checked automatically 88.1% Not in cards
Harvey LAB-AA v1.0 Artificial Analysis runs agents on Harvey's Legal Agent Benchmark tasks in an offline sandbox, where they write legal work products such as memos and redlines from case documents. Judges grade each deliverable against a task rubric, and the headline is the share of rubric criteria passed. Expert Graded by a model 94.6% In use
MBE (Multistate Bar Examination) Models answer the 200 multiple choice questions of an official Multistate Bar Examination practice exam. Scored by accuracy. Expert Checked automatically 85% System cards only
RedlineBench Agent acts as counsel at one turn of a SaaS contract negotiation and returns tracked-change redlines. Three LLM judges grade attorney-written rubrics. Expert Graded by a model 50.5% Not in cards
Uniform Bar Exam (MBE+MEE+MPT) Models take a simulated Uniform Bar Exam with multiple-choice, essay and performance test parts. Scored on the 400-point scale. Expert Graded by people 298.0 System cards only
CaseHOLD Models read a passage from a US court decision with the holding of a cited case masked out and pick the correct holding from five choices. The dataset has more than 53,000 questions drawn from US case law. Expert Checked automatically Not in cards
CUAD Models review commercial contracts and highlight the clauses a lawyer would flag, across 41 categories such as governing law, non-compete and change of control. Legal experts from The Atticus Project wrote more than 13,000 labels over 510 contracts. Scores measure how well the extracted spans match the expert labels. Expert Checked automatically Not in cards
LAB: Contracts 500 in-house contracting tasks (drafting, redlines, issues lists, escalation memos) across 10 contract categories. All-pass expert-rubric grading. Expert Graded by a model Not in cards
LAB: Diligence Agent researches a synthetic M&A virtual data room (thousands of documents, up to ~80M tokens) and drafts a diligence memo scored against planted-issue rubric criteria. Expert Graded by a model Not in cards
LAB: US Case Law Agents draft a litigation brief for the same party and motion as a real filed federal brief, from the case record. Model judges compare briefs pairwise and wins become Elo ratings. Expert Graded by a model Not in cards
LexGLUE Models solve a suite of English legal text tasks drawn from court cases, EU law, contracts and terms of service. Most tasks are classification, and one asks for the holding of a US case. Each task reports micro and macro F1. Expert Checked automatically Not in cards
StrataBench Land v1 Models classify land instruments, extract lease terms, and distinguish petroleum terms in three separate benchmarks. Classify and Extract have their own records. Define publishes three measures without one headline score. Expert Checked automatically Not in cards
StrataBench Land v1 Classify Models identify the legal act performed by each of 66 synthetic land instruments. Collide scores label accuracy under open and closed prompt protocols. Expert Checked automatically Not in cards

Finance

BenchmarkLevelScoredBest scoreStatus
TaxEval v2 Models answer expert-written tax questions. A judge grades answer correctness and stepwise reasoning against expert solutions. Expert Graded by a model 80.4% Not in cards
CorpFin v2 Models answer expert-written questions about credit agreements that often exceed 200 pages. An LLM judge grades each answer for accuracy. Expert Graded by a model 73.2% In use
Vals Finance Agent v2 927 expert-reviewed research questions on public-company filings. Partial credit only after required checks pass. Private test split. Expert Graded by a model 65.4% In use
MortgageTax Vision models read mortgage tax certificates given as images. They extract the year, parcel number and county, and compute the annualized amount due. Everyday Checked automatically 72.1% Not in cards
Tax Agent Bench Agents research 391 expert-written US corporate tax questions with tools and submit one cited written answer. A judge grades it against a rubric and checks that citations are real. Expert Graded by a model 52.3% In use
Excel Modeling Benchmark Agents build investment banking and private equity financial models in Excel from a written brief and source files. Rubrics give partial credit for tabs, structure, formulas and formatting. Expert Graded by a model 76.7% Not in cards
Vals Finance Agent Agents answer 537 financial analyst questions on public company SEC filings with search and filing tools. A three-vote GPT-5.2 judge grades each final answer. Expert Graded by a model 64.4% Fading
PolicyBench No-tools estimate of US taxes and benefits for 100 households across 18 programs. Headline metric is answers within $1 of PolicyEngine-US. Expert Checked automatically 95.2% Not in cards
Vals Finance Agent v1 Agents answer 537 financial analyst questions on public company SEC filings using search and filing tools. An LLM judge grades each final answer. Expert Graded by a model 60.6% Not in cards
Excel Modeling Benchmark v1.1 Agents build the same 103 Excel financial modeling tasks as EMB v1.0, with the Scratch rubric judge changed to GPT-6 Luna. Expert Graded by a model 77.4% In use
Vals Finance Agent v2.1 Agents answer analyst-grade questions on public company filings, scored with dealbreaker-gated partial credit. A three-model jury grades answers and fails hedged ones. Expert Graded by a model 69.5% In use
TaxCalcBench v2 Models read W-2, 1099 and other PDFs and compute full federal and state returns line by line. A return counts only when every evaluated line is exact. Expert Checked automatically 83.7% Not in cards
SnorkelUnderwrite Agents help simulated small business insurance underwriters across six task types in multi-turn chats, using tools and asking the underwriter for missing facts. Expert Graded by a model 91% Not in cards
DeepFinance Agents do private equity and investment banking work on 21 anonymized real deals: valuation, due diligence, underwriting and financial modeling. Each task comes with a data room of about 19 files and would take a professional about 27 hours. Rubrics of about 84 criteria grade the output, the workflow and the judgment behind each assumption. Expert Graded by a model 57.6% Not in cards
FinQA (Snorkel OpenEnv environment) Agent answers 290 multi-step questions on SEC 10-K data via MCP tools and constrained SQL. Binary reward with 1% numeric tolerance. University Checked automatically 81% Not in cards
SnorkelFinance ReAct agents answer expert-verified financial questions built from 10-K filings, using SQL, code and other MCP tools without seeing the documents. A judge scores full traces. University Graded by a model 81% Not in cards
Finance Reasoning (Snorkel) Agents answer expert-written financial reasoning questions grounded in 10-K tables loaded into a database, using SQL tools over many steps. A judge scores correctness and completeness. Expert Graded by a model 53.1% Not in cards
FrontierFinance Agents answer 220 investment research queries from finance experts across six use cases of the investor workflow. Three judge models check each answer against 11,543 source-attributed rubrics. Expert Graded by a model 55.9% In use
DAYJOB: Accounting Agents do accounting assignments from a short request and a workspace of files. An agent judge applies an expert rubric, and a pass needs 95% of criteria. Expert Graded by a model 35.2% Not in cards
TaxCalcBench Models compute complete US federal tax returns line by line from taxpayer data, with no tax engine. A return counts only when every evaluated line is exact. Expert Checked automatically 62.8% Not in cards
SnorkelFinance 2.0 Agents do 200 investment banking and private equity workflow tasks in a simulated firm with incomplete requests, scattered evidence, compliance rules and required state changes. Expert Graded by a model 25.1% Not in cards
SnorkelUnderwrite 2.0 Agents do 200 commercial property and casualty underwriting tasks in a simulated insurer with partial information, role permissions and stateful workflows. Expert Graded by a model 30.5% Not in cards
DAYJOB: Tax Agents take tax assignments from client request to finished return, workpaper or advice, using messy records. A pass needs 95% of the expert rubric criteria. Expert Graded by a model 10.8% Not in cards
FinSearchComp Agents search the web to fetch live financial data, look up history, and run multi-step investigations, across 635 expert-written questions. Scored by answer accuracy. Expert Graded by a model 81.7% In use
BizFinBench Models answer 6,781 Chinese finance queries drawn from real business scenarios. The queries cover numerical calculation, reasoning, information extraction, prediction recognition and knowledge questions, in nine categories. Some tasks are scored against answers and others by IteraJudge, the authors' LLM judging method. University 68.6% Not in cards
StockBench Agents trade stocks over a fixed evaluation window. StockBench ranks models by the mean of final return, maximum drawdown and Sortino ratio. The score is the board's rank, so a lower score is better. Expert Settled by events 1.00 Not in cards
ATLAS-Finance Agents do 100 investment banking tasks inside 13 simulated firms. They gather facts from email, chat, calendar and files over MCP, then build an Excel model graded against expert rubrics. Expert Graded by a model 10.7% Not in cards
ATLAS-Finance v1.1.0 Agents do the same 100 ATLAS-Finance banking tasks, graded with revised rubrics. Weights use a 1 to 10 scale and tolerances and gates sit at the section level. Expert Graded by a model 10.7% Not in cards
BankerToolBench v1.1 Agents complete 100 junior investment banker tasks and produce Excel, PowerPoint and Word deliverables. A task passes only if every critical rubric criterion passes. Expert Graded by a model 32.6% Not in cards
Hebbia Financial Services Benchmark Agents in Hebbia's production harness answer questions over deal and research documents, build Excel models and make client decks. A run passes only if it meets every rubric criterion. Expert Graded by a model 74.6% Not in cards
Investment Banking Modeling Tasks (OpenAI) Models build first- to third-year investment banking analyst models, such as three-statement and leveraged buyout models, with formatting and citations. OpenAI grades the outputs internally. Expert Graded by a model 88.6% Fading
Investor's Last Exam Financial agents make equity investment judgments from a frozen set of decision-time evidence. Predictions are scored against later outcomes and professional criteria. Expert Graded by people 68.1 Not in cards
BizFinBench v2 Models answer expert financial questions grounded in business data. The score is average accuracy on eight offline tasks. University Graded by a model 61.5% Not in cards
MultiFinBen Models complete 36 financial datasets across five languages and text, vision, and audio. The board reports a modality-balanced score. University 33.5 Not in cards
QF-Bench Agents write and run code for 87 quantitative finance tasks in a network-isolated sandbox with real data. Tasks cover derivatives pricing, fixed income, credit, risk, factor research, backtesting and market microstructure. More than 3,300 assertions check numeric outputs against finance rules such as no-arbitrage bounds and put-call parity. Scored by pass@1 over three runs. Expert Checked automatically 61.7% Not in cards
Quant-Arena Models act as quant researchers on 2022 to 2023 China A-share data with bounded feedback, then commit one strategy. A cash-account evaluator with trading costs scores it once on a sealed 2024 window. Expert Checked automatically 51.3 Not in cards
CFLUE Knowledge Models answer Chinese financial qualification questions. The score is answer accuracy across the knowledge test. University Checked automatically 53.8% Not in cards
Finch (FinWorkBench) Agent completes 172 enterprise finance and accounting spreadsheet workflows (384 tasks) from real data. Human plus calibrated LLM-judge grading. Expert Graded by people 43% In use
SheetBench Agents solve finance and accounting spreadsheet tasks through a browser in a fresh Google Sheet. The score is the share of expected answer cells that match exactly. Expert Checked automatically 64.7% Not in cards
FinanceReasoning Models solve financial numerical reasoning problems that need one or more finance formulas. Answers are numbers checked against reference Python solutions. University Checked automatically 86.3% System cards only
FinEval Models answer 8,351 Chinese finance questions in four areas: academic knowledge in 34 subjects, industry practice, financial security and financial agent tasks. Most questions are multiple choice, and the agent part tests tool use and multi-step reasoning. The makers run zero-shot and five-shot chain-of-thought settings. University Checked automatically 69.4% Not in cards
BankerToolBench 100 end-to-end junior-banker tasks producing Excel, PowerPoint and Word deliverables. Agentic grader scores ~150 weighted expert rubric criteria. Expert Graded by a model 78.6% In use
FinStepBench CorporateValuation Models value companies using financial evidence and calculations. The score is the published CorporateValuation percentage. Expert Graded by a model 77.3% Not in cards
FinStepBench DeepResearch Models write finance research reports. The score is the published DeepResearch percentage. Expert Graded by a model 59.1% Not in cards
FinStepBench LiveSearch Models retrieve and verify live financial facts. The score is the published LiveSearch percentage. Expert Graded by a model 76.7% Not in cards
KBank-MMLU (Upstage) Models answer Korean finance questions in a multiple-choice format. Upstage built the set in house. Expert Checked automatically 80.8% Fading
MBABench Agents build complete financial models in Excel from a case prompt and documents. An LLM judge grades accuracy, formulas and formatting on a 0 to 100 scale. Expert Graded by a model 31.2 Not in cards
MBABench 2.0 Agents build spreadsheet financial models for 101 tasks written by finance experts. A task passes only if every number matches the reference and the workbook meets a quality bar. Expert Checked automatically 15.8% Not in cards
BigFinanceBench Agents answer workflow-grounded financial research questions with web search, SEC EDGAR, page fetch and Python. A judge grades each answer against an expert rubric and reference. Expert Graded by a model 51% In use
CFLUE Application Models complete Chinese financial language tasks. The score is the published average of the application task metrics. University Checked automatically 44.2 Not in cards
FinEval 6.0 Models solve financial evaluation tasks. The score is the official 6.0 total score. University Checked automatically 80.9% Not in cards
Real-World Finance (Anthropic) Agents complete about 50 analyst tasks in banking, private equity, hedge funds and corporate finance, producing spreadsheets, slides and documents. Rubric graded. Expert Graded by a model 70% System cards only
Real-World Finance v2 (Anthropic) Agents complete 294 long-horizon finance tasks such as building financial models. A model grader compares deliverables pairwise and ratings are fit as Elo. Expert Graded by a model 1,637 System cards only
DiligenceBench 150 public-equity research tasks on large US companies across sectors. Agents answer open-ended research questions in their own harness, and a model judge scores each answer against a dense task rubric for factual accuracy, analysis and risk awareness. Expert Graded by a model 58.1% In use
Finance Modeling Agents complete expert-written financial modeling problems with Excel tools, from research to analysis. Expert-designed rubrics score each problem, averaged over three runs. Expert Graded by a model 21.6% Fading
Finance Modeling Pro Agents complete financial modeling tasks in Excel. Expert rubrics score each task over three trials. Expert Graded by a model 57% System cards only
FinSearchComp Global Agents search for financial information about global assets across three task types. The score is the published average accuracy for the Global subset. Expert Graded by a model 65.5% Dropped
FinStepBench Models do three kinds of finance work: retrieving and checking live financial facts, building company valuations, and writing full research reports. Graders check accuracy, sourcing and reproducible calculations. Expert Graded by a model 63.6% In use
Hy-FinAgentBench Agents complete Tencent in-house finance analyst tasks. Task and scoring details are not published. Expert Checked automatically 79.7% In use
Hy-FinModelBench Agents build financial models for Tencent in-house finance tasks. Task and scoring details are not published. Expert Checked automatically 69% In use
Hy-FinmodelBench v2 Agents build financial models for a second version of Tencent in-house finance tasks. Task and scoring details are not published. Expert Checked automatically 57% In use
ConvFinQA Models answer a conversation of related numerical questions about a company's financial report, with tables and text, where later questions build on earlier answers. The dataset has about 3,900 conversations with 14,000 questions, built from FinQA. Answers are checked by execution accuracy against the gold reasoning program. University Checked automatically 58.7% Not in cards
Finance Bench (Moonshot) Agents complete realistic financial workflows end to end, from source materials to deliverables that reviewers can check. Expert Graded by a model 62.6% System cards only
FinanceBench Models answer open-book questions about public companies using their filings, such as 10-Ks and earnings reports. The full set has 10,231 questions with answers and evidence strings, and 150 of them are public. The makers graded answers by hand, and later users often grade with a model judge. Expert Graded by people 41% Not in cards
AGORA (Agentic Global Open Research Arena) Autonomous AI systems get one secret open finance question, 72 hours and a fixed budget to write a paper. Five finance scholars judge it. Research frontier Graded by people Not in cards
Alpha Arena Frontier models trade live markets with real money over a season, and the leaderboard ranks them by returns. Expert Settled by events Not in cards
CFLUE Models answer Chinese financial questions in two parts. The knowledge part has more than 38,000 multiple-choice questions from financial qualification exams, with explanations to test reasoning. The application part has more than 16,000 test items for classification, translation, relation extraction, reading comprehension and text generation. University Checked automatically Not in cards
ClaimBench Models label whether a 10-K excerpt supports, contradicts or says nothing about a claim, across 1,960 claims. Small decision models are scored on accuracy, cost and latency. University Checked automatically Not in cards
FailSafeQA Models answer finance questions over long documents while the query or the document is degraded, irrelevant or missing. An LLM judge scores robustness, context grounding and compliance. Expert Graded by a model Not in cards
FinBen A finance suite of 35 datasets across 23 tasks, in three difficulty levels. Tasks include sentiment analysis, information extraction, numerical question answering, text generation, forecasting and stock trading. University Not in cards
FinQA Models answer numerical questions about company earnings reports that mix tables and text. Financial experts wrote 8,281 question and answer pairs, each with a gold reasoning program. Answers are checked by execution accuracy and program accuracy. University Checked automatically Not in cards
FinSearchComp Greater China Agents search for financial information about Greater China assets across three task types. The score is the published average accuracy for the Greater China subset. Expert Graded by a model Not in cards
LiveMACE Agents trade in live financial markets, and the benchmark scores the process capabilities behind their decisions, not only returns. Expert Checked automatically Not in cards
Moneda AdviceBench Models answer personal finance requests and paired education questions. Moneda’s default board scores paired pass rate on 50 EU personal requests with education twins. Expert Graded by a model Not in cards
Moneda Bench Models answer finance questions across seven separate benchmarks. AdviceBench, CiteBench, FilingsBench, GroundBench and PortfolioBench have their own records. KnowledgeBench and MarketBench have no published results yet. Moneda publishes no combined score. Expert Graded by a model Not in cards
Moneda CiteBench Models write company research notes with linked sources. Moneda scores the weighted share of linked claims that the cited pages support across 100 notes. Expert Graded by a model Not in cards
Moneda FilingsBench Models answer questions about SEC filings without receiving documents. Moneda scores accuracy on the private questions using filing data and judges for segment questions. Expert Graded by a model Not in cards
Moneda GroundBench Models answer live questions about market figures. Moneda’s default board reports the share of correct answers against reference data, with incorrect answers reported separately. Expert Graded by a model Not in cards
Moneda PortfolioBench Models solve hypothetical portfolio tasks involving rebalancing, risk, position sizing, and constraints. Moneda’s default board scores accuracy on 150 tasks with fixed answers. Expert Graded by a model Not in cards
TAT-QA Models answer 16,552 questions about 2,757 passages from real financial reports that mix a table with text. Many answers need arithmetic, counting or comparison. Scored by exact match and F1. University Checked automatically Not in cards

Medicine

BenchmarkLevelScoredBest scoreStatus
MedScribe Models write SOAP notes from clinical visit transcripts. Notes are graded against expert gold-standard rubrics for required content and section structure. Expert Graded by a model 91.4% Not in cards
MedCode Models read de-identified patient records and output primary and secondary ICD-10-CM diagnosis codes. Answers are checked against codes assigned by professional coders. Expert Checked automatically 63.6% Not in cards
MedQA Models answer multiple-choice questions from medical licensing exams, usually the 12,723-question English USMLE set. Scoring is accuracy. Expert Checked automatically 86.3% Dropped
MedXpertQA-MM Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy. Expert Checked automatically 80.4% In use
HealthBench 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. Expert Graded by a model 67.2% In use
HealthBench Professional 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. Expert Graded by a model 69.2% In use
MedXpertQA MM Extended Models answer multimodal medical questions over clinical images and patient details. Mercor adds 51 new expert-written tasks to the 200 public ones and reports both scores. Expert Checked automatically 71.2% Not in cards
BRIDGE Models work on 87 tasks built from real clinical text, such as electronic health records, in nine languages. The tasks span triage, consultation, information extraction, diagnosis, prognosis and billing codes across 14 specialties. Each task uses its own automatic metric, and the board reports the average. Expert Checked automatically 56.0 Not in cards
DAYJOB: Healthcare Agents handle short requests from healthcare staff using charts, claims exports, policies and emails. An agent judge applies an expert rubric, and a pass needs 95% of criteria. Expert Graded by a model 48.3% Not in cards
HealthBench Hard Models answer the 1,000 hardest HealthBench conversations, graded against physician-written rubrics. Expert Graded by a model 46.2% Fading
CHI-Bench Agents run long healthcare operations workflows such as prior authorization through 87 MCP tools over 20 simulated apps. Tasks need policy lookups, role handoffs and multi-turn dialogs. Expert Checked automatically 54.7% Not in cards
PMC-VQA Models answer multiple-choice questions about medical images drawn from PubMed Central figures. Scored by accuracy. University Checked automatically 66.2% Fading
RCM-Bench-1 Agents handle healthcare revenue cycle work, such as claim submission, billing and eligibility checks, with tools. Checks cover tool calls, arguments, replies and final system state. Expert Checked automatically 92.7% Not in cards
SLAKE Models answer English and Chinese questions about radiology images of the head, neck, chest, abdomen, and pelvis. Scoring is accuracy. University Checked automatically 90.8% Fading
HealthBench Consensus Models answer HealthBench conversations graded only on criteria that several physicians agreed on. Expert Graded by a model 96.6% Dropped
LifeSci Diamond Models write regulatory affairs deliverables for drugs, biologics and medical devices, such as change assessment memos, dossier inconsistency logs and agency letters. The 55 tasks span five families: reporting categories, Module 2/3 dossier consistency, missing evidence, submissions and correspondence, and device vigilance. A model judge grades each answer against a task rubric, and a task passes only when every rubric line is met and no penalty fires. Expert Graded by a model 29.1% Not in cards
Bedside Bench Models answer 500 physician-written clinical cases on diagnosis, workup, escalation and medication safety. An LLM judge grades each answer against the authors' criteria. Expert Graded by a model 94.6% Not in cards
PhysicianBench 100 physician tasks performed by an agent inside an EHR environment. Scored by pass rate per attempt. Expert Checked automatically 50% System cards only
GMAI-MMBench Vision-language models answer medical multiple-choice questions over many imaging modalities, clinical tasks and departments, at several perceptual granularities. Expert Checked automatically 43.4% Not in cards
MedHELM Models run a suite of medical tasks chosen by clinicians across five categories, from clinical decision support and note writing to patient messages and administration. Expert Graded by a model 70.3 Not in cards
PharmaBench (micro1) Agents write clinical trial documentation inside 8 synthetic drug programs, across 120 tasks written by biostatisticians, medical writers, safety physicians and regulatory staff. Expert Graded by a model 27.5% Not in cards
RealmPathology Models interpret real anatomic pathology reports and must not claim a diagnosis, stage or treatment the report does not support. Rubrics have 772 positive and 299 penalty criteria. Expert Graded by a model 78% Not in cards
RadLE 2.0 Models diagnose single-image radiology cases with a confidence score from 0 to 4, or say they do not know. A confidence-weighted index rewards right answers and penalizes confident errors. Expert Checked automatically 758.0 Not in cards
KorMedMCQA Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. Expert Checked automatically 93.2% In use
MedXpertQA Models answer 4,460 expert-level medical exam questions across 17 specialties, in a text subset and an image subset. Scoring is multiple-choice accuracy. Expert Checked automatically 52.6% Fading
micro1 Medical Index Models are scored on the equal-weight average of micro1 medical benchmarks, currently RealmPathology, PharmaBench and WellnessBench. Expert Graded by a model 71% Not in cards
PatientAgentBench Patient-facing health agents talk with a simulated patient and use a sandbox of healthcare tools for scheduling, prescriptions, telehealth and escalation. An LLM jury scores over a hundred clinician-grounded criteria. Expert Graded by a model 4.25 Not in cards
PubMedQA Models answer biomedical research questions with yes, no, or maybe, given a PubMed abstract without its conclusion. Scored by accuracy. University Checked automatically 78.3% Dropped
AfriMed-QA Models answer about 15,000 open and multiple-choice medical questions from over 60 medical schools in 16 African countries. Questions span 32 specialties. University Checked automatically 75.2% Not in cards
HealthBench Consensus Global Health Models answer health conversations that need adjustment for global context such as local care access. Scored as the failure rate. Expert Graded by a model 0% System cards only
HealthBench Consensus Urgent Models answer ambiguous high-stakes health conversations validated by two or more physicians. Scored as the error rate in informing the user. Expert Graded by a model 0.40% System cards only
ScanBench Models act as doctors in 303 simulated clinical cases from 12 departments. They take a history, order lab tests and give a final diagnosis, scored per stage. Expert Graded by a model 73.8% Fading
High-risk health scenarios eval Adversarial health emergencies test whether models recognize urgency and red flags and direct users to help. Reports pass rate. Expert Graded by a model 98.9% System cards only
MedCalc-Bench Verified Models extract values from patient notes and compute results for 55 medical calculators. Verified release of MedCalc-Bench with corrected labels. Expert Checked automatically 86.2% System cards only
Medical Knowledge Self-Assessment Program (MKSAP) Models answer internal medicine board review questions from MKSAP. Scoring is accuracy. Expert Checked automatically 75% System cards only
MedMCQA Models answer multiple-choice questions from Indian medical entrance exams, AIIMS and NEET PG. Scoring is accuracy. Expert Checked automatically 77% System cards only
MedQA Mainland China Models answer mainland China medical licensing exam questions in simplified Chinese. Scoring is accuracy. Expert Checked automatically 84% System cards only
MedQA Taiwan Models answer Taiwan medical licensing exam questions in traditional Chinese. Scoring is accuracy. Expert Checked automatically 91% System cards only
MedQA USMLE (5 options) Models answer USMLE-style medical licensing questions with five answer options. Scoring is accuracy. Expert Checked automatically 86% System cards only
MMLU Anatomy Models answer MMLU multiple-choice questions from the Anatomy subject. Scoring is accuracy. University Checked automatically 89% System cards only
MMLU Clinical Knowledge Models answer MMLU multiple-choice questions from the Clinical Knowledge subject. Scoring is accuracy. Expert Checked automatically 92% System cards only
MMLU College Medicine Models answer MMLU multiple-choice questions from the College Medicine subject. Scoring is accuracy. University Checked automatically 84% System cards only
MMLU Medical Genetics Models answer MMLU multiple-choice questions from the Medical Genetics subject. Scoring is accuracy. Expert Checked automatically 96% System cards only
MMLU Professional Medicine Models answer MMLU multiple-choice questions from the Professional Medicine subject. Scoring is accuracy. Expert Checked automatically 94% System cards only
USMLE Step 1 (sample questions) Models answer the official USMLE Step 1 multiple choice sample questions. Scored by accuracy. Expert Checked automatically 68.9% System cards only
USMLE Step 2 (sample questions) Models answer the official USMLE Step 2 multiple choice sample questions. Scored by accuracy. Expert Checked automatically 63.3% System cards only
USMLE Step 3 (sample questions) Models answer the official USMLE Step 3 multiple choice sample questions. Scored by accuracy. Expert Checked automatically 67.2% System cards only
AgentClinic Doctor agents interview simulated patients, order tests and reach a diagnosis in multimodal clinical scenarios built from MedQA and NEJM cases. Expert Checked automatically Not in cards
BioASQ Systems answer biomedical questions written by experts, using PubMed articles as evidence. Questions are yes/no, factoid, list or summary types. A second task asks systems to index new PubMed articles with MeSH headings. Expert Not in cards
Chest ImaGenome Models draw bounding boxes around named anatomical regions in MIMIC-CXR chest X-rays. Scored by mean intersection over union. Expert Checked automatically Not in cards
ChestX-ray14 Models read frontal chest X-rays from the NIH Clinical Center and say whether thoracic conditions are present. Expert Checked automatically Not in cards
CheXpert Models read chest radiographs and label up to 14 observations. The test labels come from board-certified radiologists. Expert Checked automatically Not in cards
CliniCARE-Bench Agents answer a clinical question about one real MIMIC-IV patient through governed chart tools, then give a four-way verdict with cited evidence across 750 cases. Expert Graded by a model Not in cards
CT-RATE Models read 3D non-contrast chest CT volumes and say whether each of 18 conditions is present. Scored by macro accuracy or F1 against report-derived labels. Expert Checked automatically Not in cards
EHRNoteQA Models read a patient's accumulated MIMIC-IV discharge summaries and answer clinical questions about them. The set has 962 question and answer pairs, each for a different patient. Expert Checked automatically Not in cards
EHRQA (Google) Models answer questions about long synthetic outpatient health records in FHIR format. Questions cover 42 types in 10 categories and need multi-hop, temporal and arithmetic reasoning. Expert Checked automatically Not in cards
EyePACS (diabetic retinopathy grading) Models grade diabetic retinopathy in a fundus photo on a 5-level scale, posed as a 5-option question. Google's test set has 3,614 images, one per patient. Expert Checked automatically Not in cards
Health dynamic conversation eval Simulated users hide key health facts at first, and models must ask for context to reach the correct clinical conclusion. Expert Graded by a model System cards only
Health language and locale eval Models answer locale-specific health conversations in 10 languages across 25 locales, with search. Graded against clinician guidelines. Expert Graded by a model System cards only
Health static understanding eval Models answer static health questions over text conversations, documents such as lab reports, and images. Graded against clinician guidelines. Expert Graded by a model System cards only
MedAgentBench Agents use FHIR APIs in a virtual electronic health record to do 300 patient tasks in 10 categories written by physicians. Expert Checked automatically Not in cards
MIMIC-CXR (classification) Models read chest X-rays from Beth Israel Deaconess and say whether five common conditions are present. Scored by average F1 over the conditions. Expert Checked automatically Not in cards
MIMIC-CXR report generation Models write the findings of a radiology report for a MIMIC-CXR chest X-ray. Scored against the radiologist's report, most often by RadGraph F1. Expert Checked automatically Not in cards
MindEval Models hold multi-turn therapy conversations with simulated patients designed with licensed clinical psychologists. An LLM judge scores each session. Expert Graded by a model Not in cards
MindEval2 Models support simulated patients from MindSim, a tool-using harness with detailed profiles. Psychologist-written rubrics (MHQR) replace Likert scales for scoring. Expert Graded by a model Not in cards
MLCR-AA Models reason over synthetic medical claim files of about 25k to 64k tokens, as a claims reviewer would. Three judge models check completeness and accuracy. Expert Graded by a model Not in cards
MMLU Medical Models answer the multiple-choice questions from the medical subjects of MMLU, such as anatomy, clinical knowledge and professional medicine. University Checked automatically Not in cards
MS-CXR-T Models compare a current and a prior chest X-ray of one patient and say whether a finding improved, stayed stable or got worse. Expert Checked automatically Not in cards
PathMCQA (Google) Models answer 4 to 9 option questions on histopathology patches about cancer identification, grading and subtype. The test split has 450 patches from 354 slides. Expert Checked automatically Not in cards
Radiology's Last Exam (RadLE) Multimodal models make a single spot diagnosis from images alone on 50 hard radiology cases. Board-certified radiologists scored about 83%. Expert Checked automatically Not in cards
US-DermMCQA (Google) Models pick the skin condition in a teledermatology image from 4 options. The set has one image per patient from 1,996 patients and 136 conditions. Expert Checked automatically Not in cards
VQA-RAD Vision-language models answer clinicians' questions about 315 radiology images, including head CTs, MRIs and chest X-rays. Questions are closed yes or no, or open free text. Expert Checked automatically Not in cards

Other professions

BenchmarkLevelScoredBest scoreStatus
Public Benefits Bench Models answer SNAP benefit questions across seven lifecycle phases and 25 topics. A judge grades each answer against rubrics validated by SNAP policy experts. Expert Graded by a model 76.9% Not in cards
FailureSensorIQ Models answer multiple-choice questions on how industrial sensors relate to failure modes of assets. The Kaggle board shows it under IBM's AssetOpsBench citation. Expert Checked automatically 69.5% Not in cards
SnorkelManufacturing Agents make engineering, production, quality, supplier and safety decisions in a simulated discrete-parts plant across 200 tasks, a third of them CAD tasks. Expert Graded by a model 27.8% Not in cards
AssetOpsBench Agents automate industrial asset work, such as condition monitoring, maintenance planning and intervention scheduling, using sensor data, failure models and work orders. Expert Graded by a model 65% Not in cards
Education (ByteDance Seed in-house) Models solve, grade, explain and write K to 12 problems across core subjects in teaching scenarios. School Graded by a model 62.8% Fading
Advanced Sommelier (theory knowledge) Models answer wine theory questions at the advanced sommelier level. Scoring is accuracy. Expert Checked automatically 77% System cards only
Certified Sommelier (theory knowledge) Models answer wine theory questions at the certified sommelier level. Scoring is accuracy. Expert Checked automatically 86% System cards only
Introductory Sommelier (theory knowledge) Models answer wine theory questions at the introductory sommelier level. Scoring is accuracy. Expert Checked automatically 92% System cards only
IndustryBench Models answer 2,049 Chinese industrial procurement questions grounded in national standards and product records. Answers must respect regulated thresholds and safety clauses. Expert Checked automatically Not in cards