Mercor
We track 17 benchmarks that Mercor made or helped make.
3 in use 7 with public code 16 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| APEX-Agents 480 long-horizon tasks in banking, consulting and law worlds with docs, sheets, email and chat. Rubric-graded, Pass@1. | Simulated companies | Jan 2026 | 67.2% | In use |
| APEX-SWE Software-engineering productivity tasks spanning integration work and observability, such as diagnosing failures from telemetry. | Fixing real repositories | Jan 2026 | 67.6% | In use |
| APEX-Accounting Agents run month-end close tasks across 10 frozen business worlds with the apps an accountant uses. 100 tasks are graded against 1,475 rubric criteria. | Simulated companies | Jul 2026 |  71.5% | In use |
| APEX-1 Models complete expert-written work cases with source documents for four jobs: investment banking, consulting, big law and primary care. A judge model grades each response against rubrics. | Tasks from real jobs | Sep 2025 | 81.1% | Not in cards |
| APEX-Agents 1.1 Agents do long banking, consulting and law tasks in worlds with docs, sheets, email and chat. A new judge scores hedged multiple answers as zero. | Simulated companies | Sep 2026 | 90.4% | Not in cards |
| AA-LCR Extended Models answer questions that need reasoning across about 100k tokens of documents. Mercor adds 89 new expert-written tasks to the 100 public ones and reports both scores. | Reasoning over long documents | Aug 2026 | 79.5% | Not in cards |
| SWE-bench Verified Extended Agents resolve real GitHub issues by editing codebases, scored by repository tests. Mercor adds 50 new expert-written tasks to the 500 public ones and reports both scores. | Fixing real repositories | Aug 2026 |  83.3% | Not in cards |
| Terminal-Bench 2.1 Extended Agents complete long command-line tasks in containers, scored by verification tests. Mercor adds 99 new expert-written tasks to the 89 public ones and reports both scores. | Terminal tasks | Aug 2026 |  54% | Not in cards |
| GDPval Extended Models produce work products for occupations across nine GDP sectors. Mercor adds 132 new expert-written tasks to the 213 public ones and reports both scores. | Tasks from real jobs | Aug 2026 | 7.10% | Not in cards |
| BrowseComp Extended Agents browse the web to find hard-to-locate facts with short answers. Mercor adds 100 new expert-written tasks to the 130 public ones and reports both scores. | Web browsing | Aug 2026 | 73% | Not in cards |
| MedXpertQA MM Extended Models answer multimodal medical questions over clinical images and patient details. Mercor adds 51 new expert-written tasks to the 200 public ones and reports both scores. | Medicine | Aug 2026 | 71.2% | Not in cards |
| CharXiv Extended Models answer descriptive and reasoning questions about scientific charts. Mercor adds 150 new expert-written tasks to the 5000 public ones and reports both scores. | Documents, charts and text in images | Aug 2026 | 87.1% | Not in cards |
| MMMU-Pro Extended Models answer multimodal questions that need both the image and the text. Mercor adds 140 new expert-written tasks to the 1730 public ones and reports both scores. | General visual questions | Aug 2026 | 61.3% | Not in cards |
| SWE Atlas Codebase QnA Extended Agents answer questions about large codebases, graded with rubrics. Mercor adds 50 new expert-written tasks to the 124 public ones and reports both scores. | Fixing real repositories | Aug 2026 | 44.7% | Not in cards |
| HLE Extended Models answer closed-ended frontier academic questions. Mercor adds 447 new expert-written tasks to the 2158 public ones and reports both scores. | Expert-written questions | Aug 2026 | 77.3% | Not in cards |
| OWE-Bench 26 long-horizon engineering and research projects across software, physical engineering, machine learning and scientific computing, from GPU kernels to aircraft design. Agents get 4 to 18 hours per attempt, and an offline verifier scores what they ship on held-out cases. | Building whole projects | Nov 2026 | 54.6% | Not in cards |
| AI Consumer Index Models with web search handle everyday consumer requests in shopping, food, gaming and DIY. Prompt-specific rubrics with hurdle and grounding criteria grade each answer. | General assistants | Dec 2025 | Not in cards |