Surge AI
We track 18 benchmarks that Surge AI made or helped make.
8 in use 7 with public code 17 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| GDP.pdf 100 expert multimodal reasoning tasks over real economically important PDF documents. Scored by rubric-criteria pass rate. | Documents, charts and text in images | Jul 2026 |  56.4% | In use |
| Chartography 100 tasks on specialized professional charts (Kaplan-Meier, candlestick, Sankey, Bode). Answers graded against expert references. | Documents, charts and text in images | Aug 2026 | 71.6% | In use |
| Riemann-bench 25 private research-mathematics problems from professors and IMO medalists. Agents with tools, 100 runs per problem. | Research mathematics | Apr 2026 | 83.2% | In use |
| EnterpriseBench: CoreCraft Agents work inside a simulated hardware startup with more than 2,500 entities and 23 tools. They triage incidents, audit shipments and handle customers under company policy. | Simulated companies | Feb 2026 | 77.4% | In use |
| DAYJOB: Finance Agents get short requests from finance colleagues plus a workspace of filings, ledgers and emails. An agent judge applies an expert rubric, and a pass needs 95% of criteria. | Tasks from real jobs | Sep 2026 | 42.5% | In use |
| sudo L7 Coding agents get staff-level engineering requests in mostly private production codebases. Graders score whether the agent found hidden risks and shipped the right change. | Fixing real repositories | Oct 2026 | 52.2% | In use |
| HANDBOOK.md Agents do routine company work with mock email, chat, calendar, issue and commerce tools while following a 20 to 124 page handbook. Programmatic criteria check required and forbidden actions, and a trial passes only if all pass. | Following precise instructions | Jul 2026 | 52.3% | In use |
| ComplexConstraints Models answer 75 expert-written professional prompts whose constraints depend on each other, fire conditionally or must be inferred. An LLM judge grades 1,559 rubric criteria one at a time, and a task passes only if all criteria pass. | Following precise instructions | Jun 2026 | 70.8% | In use |
| Tuesday Work Index A composite score built from the Surge AI benchmark suite, such as DAYJOB, GDP.pdf, Riemann-bench and ComplexConstraints. It asks whether models get through an ordinary workday. | Composite work indices | Aug 2026 | 76.3 | Not in cards |
| Hemingway-bench Professional writers compare two models' answers blind on real writing prompts, from bedtime stories to business documents. The comparisons become ratings. | Creative writing | Feb 2026 | 1,110 | Not in cards |
| DAYJOB: Healthcare Agents handle short requests from healthcare staff using charts, claims exports, policies and emails. An agent judge applies an expert rubric, and a pass needs 95% of criteria. | Medicine | Sep 2026 | 48.3% | Not in cards |
| Antidote Experts such as doctors, lawyers and engineers compare model answers to prompts from their own history. They judge which answer would still help them a month later. | Overall chat quality | May 2026 | 1,099 | Not in cards |
| GDP.xlsx Agents open a professional Excel workbook in a sandbox and answer a short question about it, as a domain professional would. A Gemini autograder checks the final answer against criteria. | Documents and spreadsheets | Sep 2026 | 38.3% | Not in cards |
| DAYJOB: Accounting Agents do accounting assignments from a short request and a workspace of files. An agent judge applies an expert rubric, and a pass needs 95% of criteria. | Finance | Oct 2026 | 35.2% | Not in cards |
| DAYJOB: Law Agents do legal assignments from a short request and a workspace of files. An agent judge applies an expert rubric, and a pass needs 95% of criteria. | Law | Oct 2026 | 36.9% | Not in cards |
| DAYJOB: Data Science Agents do data science assignments from a short request and a workspace of files. An agent judge applies an expert rubric, and a pass needs 95% of criteria. | Data science and SQL | Oct 2026 | 37.3% | Not in cards |
| DAYJOB: Tax Agents take tax assignments from client request to finished return, workpaper or advice, using messy records. A pass needs 95% of the expert rubric criteria. | Finance | Oct 2026 | 10.8% | Not in cards |
| Surge AI blind coding evaluation Surge AI raters blindly score model coding outputs on a 1-5 scale. Mistral reports five models. | Building whole projects | Oct 2026 | Not in cards |