Benchmark Almanac

SnorkelUnderwrite

GPT-5.4 91% leads since March 2026
Not in model cards

Agents help simulated small business insurance underwriters across six task types in multi-turn chats, using tools and asking the underwriter for missing facts.

Archived. Expert-verified conversation traces. Dataset created May 22nd, 2025 and blog post June 26th, 2025.

Made by
Snorkel AI
Released
May 2025
Kind
Finance
Level
Expert
Scored
Graded by a model
Answers
Agent task
Tasks
All public
Strongest model tested
GPT-5.4 #27 on the capability index
Last model added
March 2026
Scores refreshed
November 12th, 2026
33
Entries on its own leaderboard, 27 models we track

GPT-5.4 leads with 91% among models released by March 2026

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in March 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

33 entries. Scroll sideways for the rest.

  1. 91.0 GPT-5.4
  2. 86.3 Claude Opus 4.1
  3. 83.3 GPT-5
  4. 83.3 Grok 4
  5. 81.3 Grok 4 Fast
  6. 78.0 Grok 3
  7. 78.0 o4-mini
  8. 77.0 o3
  9. 77.0 Claude Opus 4
  10. 74.6 Claude 3.7 Sonnet
  11. 72.3 Claude Sonnet 4
  12. 71.7 GPT-5 mini
  13. 71.3 Kimi K2 Thinking
  14. 70.6 GPT-4.1
  15. 61.0 Gemini 2.5 Flash [snapshot unspecified]
  16. 57.0 Amazon Nova Premier
  17. 56.3 Gemini 2.5 Pro [snapshot unspecified]
  18. 52.3 Amazon Nova Pro
  19. 47.0 GPT-5 nano
  20. 46.3 Llama 3.3 70B
  21. 46.3 Llama 4 Maverick
  22. 44.3 o3-mini
  23. 44.3 Llama 4 Scout
  24. 40.0 Amazon Nova Lite
  25. 38.3 Mistral Large [snapshot unspecified]
  26. 34.0 Codestral [snapshot unspecified]
  27. 31.0 Amazon Nova Micro
  28. 30.0 gpt-oss-120b
  29. 29.3 Magistral Medium [snapshot unspecified]
  30. 25.7 Command R+
  31. 21.3 Qwen 3 235B [snapshot unspecified]
  32. 20.0 Llama 3.1-405B
  33. 15.3 Command R (03-2024)
  1. 1 GPT-5.4 91%
  2. 2 Claude Opus 4.1 86.3%
  3. 3 GPT-5 83.3%
  4. 4 Grok 4 83.3%
  5. 5 Grok 4 Fast reasoning 81.3%
  6. 6 Grok 3 78%
  7. 7 o4-mini 78%
  8. 8 o3 77%
  9. 9 Claude Opus 4 77%
  10. 10 Claude 3.7 Sonnet 74.6%
  11. 11 Claude Sonnet 4 72.3%
  12. 12 GPT-5 mini 71.7%
  13. 13 Kimi K2 Thinking 71.3%
  14. 14 GPT-4.1 70.6%
  15. 15 Gemini 2.5 Flash [snapshot unspecified] 61%
  16. 16 Amazon Nova Premier 57%
  17. 17 Gemini 2.5 Pro [snapshot unspecified] 56.3%
  18. 18 Amazon Nova Pro 52.3%
  19. 19 GPT-5 nano 47%
  20. 20 Llama 3.3 70B 46.3%
  21. 21 Llama 4 Maverick 46.3%
  22. 22 o3-mini 44.3%
  23. 23 Llama 4 Scout 44.3%
  24. 24 Amazon Nova Lite 40%
  25. 25 Mistral Large [snapshot unspecified] 38.3%
  26. 26 Codestral [snapshot unspecified] 34%
  27. 27 Amazon Nova Micro 31%
  28. 28 gpt-oss-120b 30%
  29. 29 Magistral Medium [snapshot unspecified] 29.3%
  30. 30 Command R+ 25.7%
  31. 31 Qwen 3 235B [snapshot unspecified] 21.3%
  32. 32 Llama 3.1-405B 20%
  33. 33 Command R (03-2024) 15.3%

Source: Official SnorkelUnderwrite leaderboard

#EntryScoreSettingDateSource
1 GPT-5.4 91% Mar 2026 Official leaderboard
2 Claude Opus 4.1 86.3% Aug 2025 Official leaderboard
3 GPT-5 83.3% Aug 2025 Official leaderboard
4 Grok 4 83.3% Jul 2025 Official leaderboard
5 Grok 4 Fast 81.3% reasoning Sep 2025 Official leaderboard
6 Grok 3 78% Apr 2025 Official leaderboard
7 o4-mini 78% Apr 2025 Official leaderboard
8 o3 77% Apr 2025 Official leaderboard
9 Claude Opus 4 77% May 2025 Official leaderboard
10 Claude 3.7 Sonnet 74.6% Feb 2025 Official leaderboard
11 Claude Sonnet 4 72.3% May 2025 Official leaderboard
12 GPT-5 mini 71.7% Aug 2025 Official leaderboard
13 Kimi K2 Thinking 71.3% Nov 2025 Official leaderboard
14 GPT-4.1 70.6% Apr 2025 Official leaderboard
15 Gemini 2.5 Flash [snapshot unspecified] 61% Official leaderboard
16 Amazon Nova Premier 57% Apr 2025 Official leaderboard
17 Gemini 2.5 Pro [snapshot unspecified] 56.3% Official leaderboard
18 Amazon Nova Pro 52.3% Dec 2024 Official leaderboard
19 GPT-5 nano 47% Aug 2025 Official leaderboard
20 Llama 3.3 70B 46.3% Dec 2024 Official leaderboard
21 Llama 4 Maverick 46.3% Apr 2025 Official leaderboard
22 o3-mini 44.3% Jan 2025 Official leaderboard
23 Llama 4 Scout 44.3% Apr 2025 Official leaderboard
24 Amazon Nova Lite 40% Dec 2024 Official leaderboard
25 Mistral Large [snapshot unspecified] 38.3% Official leaderboard
26 Codestral [snapshot unspecified] 34% Official leaderboard
27 Amazon Nova Micro 31% Dec 2024 Official leaderboard
28 gpt-oss-120b 30% Aug 2025 Official leaderboard
29 Magistral Medium [snapshot unspecified] 29.3% Official leaderboard
30 Command R+ 25.7% Aug 2024 Official leaderboard
31 Qwen 3 235B [snapshot unspecified] 21.3% Official leaderboard
32 Llama 3.1-405B 20% Jul 2024 Official leaderboard
33 Command R (03-2024) 15.3% Mar 2024 Official leaderboard

4 models have held number one on the official leaderboard since its release in May 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

xAI Anthropic OpenAI
Own leaderboard Passed 85% in Aug 2025, by Claude Opus 4.1
GPT-5.4 91%

Source: Official SnorkelUnderwrite leaderboard

Claude Opus 4.1 on the official leaderboard passed 85% in August 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) GPT-5.4 Claude Opus 4.1 Grok 3 Amazon Nova Pro Command R+ Llama 3.1-405B Command R (03-2024) 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) GPT-5.4 Grok 4

Source: Official SnorkelUnderwrite leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

xAI Anthropic OpenAI
SnorkelUnderwrite 2.0 5 leaders
Grok 4.7 30.5%
SnorkelUnderwrite Passed 85% in Aug 2025, by Claude Opus 4.1
GPT-5.4 91%
SnorkelUnderwrite 2.0 1 leader
Grok 4.7 30.5%

Source: Official leaderboards

Replaced by SnorkelUnderwrite 2.0

More in Law, finance and medicine

All 179 in Law, finance and medicine
  • MedXpertQA-MM In use Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy. Qwen 3.8 Max 80.4%
  • HealthBench Professional In use 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. Claude Sonnet 5.5 69.2%
  • HealthBench In use 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. GPT-5 67.2%
  • FinSearchComp In use Agents search the web to fetch live financial data, look up history, and run multi-step investigations, across 635 expert-written questions. Scored by answer accuracy. Ling-3.1-flash 81.7%
  • KBL In use Models answer Korean legal knowledge and reasoning tasks and Korean bar exam questions. It has 7 knowledge tasks, 4 reasoning tasks and 2,510 bar exam items. Solar Pro 4 77.5%
  • KorMedMCQA In use Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. Solar Pro 4 93.2%