BigFinanceBench
Agents answer workflow-grounded financial research questions with web search, SEC EDGAR, page fetch and Python. A judge grades each answer against an expert rubric and reference.
928 questions, with a public 50-item subset in the repo. Fireworks lists 14 models on 139 private tasks.
- Made by
- Rogo Technologies
- Released
- June 2026
- Kind
- Finance
- Level
- Expert
- Scored
- Graded by a model
- Answers
- Agent task
- Tasks
- Public, plus a private test set
- Strongest model tested
- GPT-5.6 Terra #17 on the capability index
- Last in a launch post
- July 2026
- Last model added
- July 2026
In the authors’ words
Financial-research answers are decision-relevant only when another analyst can audit how they were produced: which source was chosen, which period and accounting definition were used, which assumptions were made, and how the calculation was performed. Existing finance benchmarks largely evaluate isolated subskills or final answers, leaving the auditable derivation itself under-measured. We introduce BigFinanceBench, a 928-item expert-authored benchmark of open-ended financial-research tasks in which each item pairs a ground-truth reference answer with a point-weighted rubric that decomposes the derivation into independently checkable steps. BigFinanceBench is workflow-grounded in that it evaluates the full derivation rather than only the final output. Across 36,241 rubric points, the benchmark supports partial-credit evaluation and localization of failures across the analyst workflow. Evaluating ten current frontier and open-weight agents, we find substantial headroom: the best system reaches only 58.8% rubric score, final-answer accuracy is a useful but lossy proxy for derivation quality, and model capability varies non-uniformly across financial workflows.
GPT-5.6 Terra leads with 51%
Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.
- 1 GPT-5.6 Terra 51%
- 2 GPT-5.6 Luna 36%
Source: Labs’ model cards
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-5.6 Terra | 51% | Jul 2026 | Lab model card | |
| 2 | GPT-5.6 Luna | 36% | Jul 2026 | Lab model card |
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Labs’ model cards
One lab put it in a launch post in the last six months
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Jul 9, 2026 | GPT-5.6 Luna | 36 | Big Finance Bench |
| Jul 9, 2026 | GPT-5.6 Terra | 51 | Big Finance Bench |
More in Law, finance and medicine
- MedXpertQA-MM In use Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy.
Qwen 3.8 Max 80.4%
- HealthBench Professional In use 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. Claude Sonnet 5.5 69.2%
- HealthBench In use 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. GPT-5 67.2%
- FinSearchComp In use Agents search the web to fetch live financial data, look up history, and run multi-step investigations, across 635 expert-written questions. Scored by answer accuracy.
Ling-3.1-flash 81.7%
- KBL In use Models answer Korean legal knowledge and reasoning tasks and Korean bar exam questions. It has 7 knowledge tasks, 4 reasoning tasks and 2,510 bar exam items.
Solar Pro 4 77.5%
- KorMedMCQA In use Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024.
Solar Pro 4 93.2%