Benchmark Almanac

Hebbia Financial Services Benchmark

Claude Opus 5.5 74.6% leads since October 2026
Not in model cards

Agents in Hebbia's production harness answer questions over deal and research documents, build Excel models and make client decks. A run passes only if it meets every rubric criterion.

151 private tasks from Hebbia's production work. 10 models in the Fireworks index on October 29th, 2026. No maker page with details found.

Made by
Hebbia
Released
October 2026
Kind
Finance
Level
Expert
Scored
Graded by a model
Answers
Agent task
Tasks
Kept private
Strongest model tested
GPT-6 Astra #3 on the capability index
Last model added
October 2026
Scores refreshed
November 12th, 2026
10
Models scored on its own leaderboard

Claude Opus 5.5 leads with 74.6%

Best score per model from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

  1. 74.6 Claude Opus 5.5
  2. 73.5 GPT-6.1 Sol
  3. 72.3 GPT-6 Astra
  4. 63.1 DeepSeek V4.1 Flash
  5. 60.9 Gemini 3.8 Flash
  6. 60.3 Kimi K3
  7. 60.3 DeepSeek V4.1 Flash 1017
  8. 49.0 Muse Spark Open
  9. 47.0 MiniMax-M3.1-Flash
  10. 43.7 GLM-5.3-Flash
  1. 1 Claude Opus 5.5 74.6%
  2. 2 GPT-6.1 Sol 73.5%
  3. 3 GPT-6 Astra 72.3%
  4. 4 DeepSeek V4.1 Flash 63.1%
  5. 5 Gemini 3.8 Flash 60.9%
  6. 6 Kimi K3 60.3%
  7. 7 DeepSeek V4.1 Flash 1017 60.3%
  8. 8 Muse Spark Open 49%
  9. 9 MiniMax-M3.1-Flash 47%
  10. 10 GLM-5.3-Flash 43.7%

Source: Official Hebbia Financial Services Benchmark leaderboard

#ModelScoreSettingDateSource
1 Claude Opus 5.5 74.6% Sep 2026 Official leaderboard
2 GPT-6.1 Sol 73.5% Sep 2026 Official leaderboard
3 GPT-6 Astra 72.3% Sep 2026 Official leaderboard
4 DeepSeek V4.1 Flash 63.1% Sep 2026 Official leaderboard
5 Gemini 3.8 Flash 60.9% Sep 2026 Official leaderboard
6 Kimi K3 60.3% Jul 2026 Official leaderboard
7 DeepSeek V4.1 Flash 1017 60.3% Oct 2026 Official leaderboard
8 Muse Spark Open 49% Oct 2026 Official leaderboard
9 MiniMax-M3.1-Flash 47% Oct 2026 Official leaderboard
10 GLM-5.3-Flash 43.7% Aug 2026 Official leaderboard

Strict pass rate vs. cost

The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

40% 50% 60% 70% 80% $0$1$2$3 Cost per task ($), lower is better Strict pass rate Kimi K3: Strict pass rate 60.3%, cost per task $0.82 Gemini 3.8 Flash: Strict pass rate 60.9%, cost per task $0.59 GPT-6 Astra: Strict pass rate 72.3%, cost per task $2.82 Muse Spark Open: Strict pass rate 49%, cost per task $0.44 DeepSeek V4.1 Flash 1017: Strict pass rate 60.3%, cost per task $0.12 MiniMax-M3.1-Flash: Strict pass rate 47%, cost per task $0.2 GLM-5.3-Flash: Strict pass rate 43.7%, cost per task $0.05 DeepSeek V4.1 Flash: Strict pass rate 63.1%, cost per task $0.08 Claude Opus 5.5: Strict pass rate 74.6%, cost per task $0.73 GPT-6.1 Sol: Strict pass rate 73.5%, cost per task $0.45 Claude Opus 5.5GPT-6.1 SolDeepSeek V4.1 FlashGLM-5.3-Flash 40% 50% 60% 70% 80% $0$1$2$3 Cost per task ($), lower is better Strict pass rate Kimi K3: Strict pass rate 60.3%, cost per task $0.82 Gemini 3.8 Flash: Strict pass rate 60.9%, cost per task $0.59 GPT-6 Astra: Strict pass rate 72.3%, cost per task $2.82 Muse Spark Open: Strict pass rate 49%, cost per task $0.44 DeepSeek V4.1 Flash 1017: Strict pass rate 60.3%, cost per task $0.12 MiniMax-M3.1-Flash: Strict pass rate 47%, cost per task $0.2 GLM-5.3-Flash: Strict pass rate 43.7%, cost per task $0.05 DeepSeek V4.1 Flash: Strict pass rate 63.1%, cost per task $0.08 Claude Opus 5.5: Strict pass rate 74.6%, cost per task $0.73 GPT-6.1 Sol: Strict pass rate 73.5%, cost per task $0.45 Claude Opus 5.5GPT-6.1 SolDeepSeek V4.1 FlashGLM-5.3-Flash
ModelSettingStrict pass rateCost per task
Claude Opus 5.5 74.6% $0.73
GPT-6.1 Sol 73.5% $0.45
GPT-6 Astra 72.3% $2.82
DeepSeek V4.1 Flash 63.1% $0.08
Gemini 3.8 Flash 60.9% $0.59
DeepSeek V4.1 Flash 1017 60.3% $0.12
Kimi K3 60.3% $0.82
Muse Spark Open 49% $0.44
MiniMax-M3.1-Flash 47% $0.2
GLM-5.3-Flash 43.7% $0.05

The best score, 74.6%, costs $0.73 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Z.ai DeepSeek Anthropic OpenAI Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Opus 5.5 DeepSeek V4.1 Flash GLM-5.3-Flash 0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Opus 5.5 DeepSeek V4.1 Flash

Source: Official Hebbia Financial Services Benchmark leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 5.5 74.6% $0.73
GPT-6.1 Sol 73.5% $0.45
GPT-6 Astra 72.3% $2.82
DeepSeek V4.1 Flash 63.1% $0.08
Gemini 3.8 Flash 60.9% $0.59
Kimi K3 60.3% $0.82
DeepSeek V4.1 Flash 1017 60.3% $0.12
Muse Spark Open 49% $0.44
MiniMax-M3.1-Flash 47% $0.20
GLM-5.3-Flash 43.7% $0.05

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard 1 leader
Claude Opus 5.5 74.6%

Source: Official Hebbia Financial Services Benchmark leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% Near the ceiling (85%) Claude Opus 5.5 Gemini 3.8 Flash Kimi K3 0% 20% 40% 60% 80% 100% Near the ceiling (85%) Claude Opus 5.5 Gemini 3.8 Flash

Source: Official Hebbia Financial Services Benchmark leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Law, finance and medicine

All 179 in Law, finance and medicine
  • MedXpertQA-MM In use Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy. Qwen 3.8 Max 80.4%
  • HealthBench Professional In use 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. Claude Sonnet 5.5 69.2%
  • HealthBench In use 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. GPT-5 67.2%
  • FinSearchComp In use Agents search the web to fetch live financial data, look up history, and run multi-step investigations, across 635 expert-written questions. Scored by answer accuracy. Ling-3.1-flash 81.7%
  • KBL In use Models answer Korean legal knowledge and reasoning tasks and Korean bar exam questions. It has 7 knowledge tasks, 4 reasoning tasks and 2,510 bar exam items. Solar Pro 4 77.5%
  • KorMedMCQA In use Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. Solar Pro 4 93.2%