Benchmark Almanac

Finance Reasoning (Snorkel)

Grok 4 53.1% leads since July 2025
Not in model cards

Agents answer expert-written financial reasoning questions grounded in 10-K tables loaded into a database, using SQL tools over many steps. A judge scores correctness and completeness.

Archived. Built as a harder follow-up to SnorkelFinance. Claude Sonnet 3.7 judges with the reference answer. 15 minutes and 100 turns per trace.

Made by
Snorkel AI
Released
June 2025
Kind
Finance
Level
Expert
Scored
Graded by a model
Answers
Agent task
Tasks
All public
Strongest model tested
GPT-5.4 #27 on the capability index
Last model added
March 2026
Scores refreshed
November 12th, 2026
26
Entries on its own leaderboard, 21 models we track

Grok 4 leads with 53.1% among models released by March 2026

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in March 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

26 entries. Scroll sideways for the rest.

  1. 53.1 Grok 4
  2. 52.0 GPT-5.4
  3. 51.9 Claude 3.7 Sonnet
  4. 51.0 GPT-5
  5. 49.4 Claude Sonnet 4
  6. 48.1 Claude Opus 4
  7. 46.8 Gemini 3 Pro
  8. 46.8 GPT-5 mini
  9. 45.6 o4-mini
  10. 45.6 Claude Opus 4.1
  11. 44.3 GPT-4.1
  12. 43.0 o3
  13. 41.8 Grok 3
  14. 40.5 Grok 4 Fast
  15. 35.4 Llama-3.3-Nemotron-Super-49B-v1.5
  16. 35.0 Kimi K2 Thinking
  17. 34.6 Gemini 2.5 Pro [snapshot unspecified]
  18. 34.2 Amazon Nova Premier
  19. 32.0 Gemini 2.5 Flash [snapshot unspecified]
  20. 31.6 gpt-oss-120b
  21. 30.4 o3-mini
  22. 26.6 GPT-5 nano
  23. 17.7 Qwen 3 235B [snapshot unspecified]
  24. 13.9 Magistral Medium [snapshot unspecified]
  25. 12.7 Amazon Nova Pro
  26. 10.1 Mistral Large [snapshot unspecified]
  1. 1 Grok 4 53.1%
  2. 2 GPT-5.4 52%
  3. 3 Claude 3.7 Sonnet 51.9%
  4. 4 GPT-5 51%
  5. 5 Claude Sonnet 4 49.4%
  6. 6 Claude Opus 4 48.1%
  7. 7 Gemini 3 Pro 46.8%
  8. 8 GPT-5 mini 46.8%
  9. 9 o4-mini 45.6%
  10. 10 Claude Opus 4.1 45.6%
  11. 11 GPT-4.1 44.3%
  12. 12 o3 43%
  13. 13 Grok 3 41.8%
  14. 14 Grok 4 Fast reasoning 40.5%
  15. 15 Llama-3.3-Nemotron-Super-49B-v1.5 35.4%
  16. 16 Kimi K2 Thinking 35%
  17. 17 Gemini 2.5 Pro [snapshot unspecified] 34.6%
  18. 18 Amazon Nova Premier 34.2%
  19. 19 Gemini 2.5 Flash [snapshot unspecified] 32%
  20. 20 gpt-oss-120b 31.6%
  21. 21 o3-mini 30.4%
  22. 22 GPT-5 nano 26.6%
  23. 23 Qwen 3 235B [snapshot unspecified] 17.7%
  24. 24 Magistral Medium [snapshot unspecified] 13.9%
  25. 25 Amazon Nova Pro 12.7%
  26. 26 Mistral Large [snapshot unspecified] 10.1%

Source: Official Finance Reasoning (Snorkel) leaderboard

#EntryScoreSettingDateSource
1 Grok 4 53.1% Jul 2025 Official leaderboard
2 GPT-5.4 52% Mar 2026 Official leaderboard
3 Claude 3.7 Sonnet 51.9% Feb 2025 Official leaderboard
4 GPT-5 51% Aug 2025 Official leaderboard
5 Claude Sonnet 4 49.4% May 2025 Official leaderboard
6 Claude Opus 4 48.1% May 2025 Official leaderboard
7 Gemini 3 Pro 46.8% Nov 2025 Official leaderboard
8 GPT-5 mini 46.8% Aug 2025 Official leaderboard
9 o4-mini 45.6% Apr 2025 Official leaderboard
10 Claude Opus 4.1 45.6% Aug 2025 Official leaderboard
11 GPT-4.1 44.3% Apr 2025 Official leaderboard
12 o3 43% Apr 2025 Official leaderboard
13 Grok 3 41.8% Apr 2025 Official leaderboard
14 Grok 4 Fast 40.5% reasoning Sep 2025 Official leaderboard
15 Llama-3.3-Nemotron-Super-49B-v1.5 35.4% Jul 2025 Official leaderboard
16 Kimi K2 Thinking 35% Nov 2025 Official leaderboard
17 Gemini 2.5 Pro [snapshot unspecified] 34.6% Official leaderboard
18 Amazon Nova Premier 34.2% Apr 2025 Official leaderboard
19 Gemini 2.5 Flash [snapshot unspecified] 32% Official leaderboard
20 gpt-oss-120b 31.6% Aug 2025 Official leaderboard
21 o3-mini 30.4% Jan 2025 Official leaderboard
22 GPT-5 nano 26.6% Aug 2025 Official leaderboard
23 Qwen 3 235B [snapshot unspecified] 17.7% Official leaderboard
24 Magistral Medium [snapshot unspecified] 13.9% Official leaderboard
25 Amazon Nova Pro 12.7% Dec 2024 Official leaderboard
26 Mistral Large [snapshot unspecified] 10.1% Official leaderboard

2 models have held number one on the official leaderboard since its release in June 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic xAI
Own leaderboard Not tested since Mar 2026
Grok 4 53.1%

Source: Official Finance Reasoning (Snorkel) leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Grok 4 Claude 3.7 Sonnet o3-mini Amazon Nova Pro 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Grok 4 o3-mini

Source: Official Finance Reasoning (Snorkel) leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Law, finance and medicine

All 179 in Law, finance and medicine
  • MedXpertQA-MM In use Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy. Qwen 3.8 Max 80.4%
  • HealthBench Professional In use 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. Claude Sonnet 5.5 69.2%
  • HealthBench In use 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. GPT-5 67.2%
  • FinSearchComp In use Agents search the web to fetch live financial data, look up history, and run multi-step investigations, across 635 expert-written questions. Scored by answer accuracy. Ling-3.1-flash 81.7%
  • KBL In use Models answer Korean legal knowledge and reasoning tasks and Korean bar exam questions. It has 7 knowledge tasks, 4 reasoning tasks and 2,510 bar exam items. Solar Pro 4 77.5%
  • KorMedMCQA In use Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. Solar Pro 4 93.2%