Benchmark Almanac

Loss Run Extraction (Bevaya)

Our Loss Run Model 93.1% tops its leaderboard Claude Fable 5 85.3% leads the models we track
Not in model cards

Models extract claims and policy tables from insurance loss reports into JSON. The main metric counts documents where every field matches the reference.

Bevaya compares its model with 12 frontier models on 346 documents. The dashboard also reports field accuracy, latency, and cost.

Made by
Bevaya Labs
Released
August 2026
Kind
Documents, charts and text in images
Level
University
Scored
Checked automatically
Answers
Open-ended
Tasks
Kept private
Measures
Capability
Language
English
In and out
Documents → Tables
Interaction
Single turn
Items from
Real work and use
Refresh
One fixed set
Run by
A host
In views
Law and finance
Strongest model tested
Claude Opus 5 #11 on the capability index
Last model added
July 2026
Scores refreshed
November 20th, 2026
13
Entries on its own leaderboard, 12 models we track

Our Loss Run Model tops the leaderboard with 93.1% Claude Fable 5 leads with 85.3% among models released by July 2026

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Claude Fable 5, at 85.3%. This view shows only models we track, each at its best.

The newest model with a score here came out in July 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 Our Loss Run Model serial 93.1%
  2. 2 Claude Fable 5 medium thinking 85.3%
  3. 3 Claude Opus 5 medium thinking 85.3%
  4. 4 Gemini 3.6 Flash medium thinking 83.7%
  5. 5 Gemini 3.1 Pro non-thinking 83.5%
  6. 6 Claude Sonnet 5 medium thinking 83.4%
  7. 7 Claude Opus 4.8 medium thinking 83.2%
  8. 8 Claude Sonnet 4.6 non-thinking 82.9%
  9. 9 Gemini 3.5 Flash medium thinking 82.6%
  10. 10 GPT-5 non-thinking 82.1%
  11. 11 GPT-4.1 non-thinking 80.4%
  12. 12 Gemini 3.1 Flash-Lite non-thinking 80.1%
  13. 13 Claude Haiku 4.5 non-thinking 77.7%

Source: Official Loss Run Extraction (Bevaya) leaderboard

#EntryScoreSettingDateSource
1 Our Loss Run Model 93.1% serial Official leaderboard
2 Claude Fable 5 85.3% medium thinking Jun 2026 Official leaderboard
3 Claude Opus 5 85.3% medium thinking Jul 2026 Official leaderboard
4 Gemini 3.6 Flash 83.7% medium thinking Jul 2026 Official leaderboard
5 Gemini 3.1 Pro 83.5% non-thinking Feb 2026 Official leaderboard
6 Claude Sonnet 5 83.4% medium thinking Jun 2026 Official leaderboard
7 Claude Opus 4.8 83.2% medium thinking May 2026 Official leaderboard
8 Claude Sonnet 4.6 82.9% non-thinking Feb 2026 Official leaderboard
9 Gemini 3.5 Flash 82.6% medium thinking May 2026 Official leaderboard
10 GPT-5 82.1% non-thinking Aug 2025 Official leaderboard
11 GPT-4.1 80.4% non-thinking Apr 2025 Official leaderboard
12 Gemini 3.1 Flash-Lite 80.1% non-thinking Mar 2026 Official leaderboard
13 Claude Haiku 4.5 77.7% non-thinking Oct 2025 Official leaderboard
#ModelScoreSettingDateSource
1 Claude Fable 5 85.3% medium thinking Jun 2026 Official leaderboard
2 Claude Opus 5 85.3% medium thinking Jul 2026 Official leaderboard
3 Gemini 3.6 Flash 83.7% medium thinking Jul 2026 Official leaderboard
4 Gemini 3.1 Pro 83.5% non-thinking Feb 2026 Official leaderboard
5 Claude Sonnet 5 83.4% medium thinking Jun 2026 Official leaderboard
6 Claude Opus 4.8 83.2% medium thinking May 2026 Official leaderboard
7 Claude Sonnet 4.6 82.9% non-thinking Feb 2026 Official leaderboard
8 Gemini 3.5 Flash 82.6% medium thinking May 2026 Official leaderboard
9 GPT-5 82.1% non-thinking Aug 2025 Official leaderboard
10 GPT-4.1 80.4% non-thinking Apr 2025 Official leaderboard
11 Gemini 3.1 Flash-Lite 80.1% non-thinking Mar 2026 Official leaderboard
12 Claude Haiku 4.5 77.7% non-thinking Oct 2025 Official leaderboard

Rankings by part

We read 2 parts from the benchmark’s own leaderboard: Claims and Policy. Each ranks models on one part of the test. Claude Opus 5 leads Claims and Claude Fable 5 leads Policy. Arrows show how far a model moved from its overall rank among the models we track.

Claims

12 models we track have a score here.

  1. 1 Claude Opus 5 ↑1 87.3%
  2. 2 Claude Fable 5 ↓1 85.6%
  3. 3 Claude Opus 4.8 ↑3 85.3%
  4. 4 Claude Sonnet 4.6 ↑3 84.9%
  5. 5 Gemini 3.1 Pro ↓1 84.8%
  6. 6 Gemini 3.6 Flash ↓3 84.8%
  7. 7 Claude Sonnet 5 ↓2 84.7%
  8. 8 Gemini 3.5 Flash 83.1%
  9. 9 Gemini 3.1 Flash-Lite ↑2 83.1%
  10. 10 GPT-5 ↓1 82.8%

The best score, 85.3%, costs $0.46 a task

Runs on the cost frontier score higher than any run that's cheaper. GPT-4.1 at non-thinking comes within 4.9 points of the best for $0.05 a task, 10 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI Google DeepMind Anthropic Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.001 $0.01 $0.10 $1.00 Cost per task, log scale Claude Fable 5, medium thinking Gemini 3.1 Pro, non-thinking Gemini 3.1 Flash-Lite, non-thinking 0% 20% 40% 60% 80% 100% $0.001 $0.01 $0.10 $1.00 Cost per task, log scale Claude Fable 5, medium thinking

Source: Official Loss Run Extraction (Bevaya) leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Fable 5 medium thinking 85.3% $0.46
Claude Opus 5 medium thinking 85.3% $0.19
Gemini 3.6 Flash medium thinking 83.7% $0.15
Gemini 3.1 Pro non-thinking 83.5% $0.07
Claude Sonnet 5 medium thinking 83.4% $0.15
Claude Opus 4.8 medium thinking 83.2% $0.24
Claude Sonnet 4.6 non-thinking 82.9% $0.09
Claude Opus 4.8 non-thinking 82.9% $0.17
Gemini 3.6 Flash non-thinking 82.8% $0.05
Claude Sonnet 5 non-thinking 82.7% $0.21
Gemini 3.5 Flash medium thinking 82.6% $0.20
Gemini 3.1 Pro medium thinking 82.4% $0.12
GPT-5 non-thinking 82.1% $0.15
Gemini 3.5 Flash non-thinking 82% $0.05
GPT-4.1 non-thinking 80.4% $0.05
Gemini 3.1 Flash-Lite non-thinking 80.1% $0.008
Gemini 3.1 Flash-Lite medium thinking 79.9% $0.01
Claude Haiku 4.5 non-thinking 77.7% $0.03

The best score, 85.3%, costs $0.46 a task

Runs on the cost frontier score higher than any run that's cheaper. GPT-4.1 at non-thinking comes within 4.9 points of the best for $0.05 a task, 10 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI Google DeepMind Anthropic Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.001 $0.01 $0.10 $1.00 Cost per task, log scale Claude Fable 5, medium thinking Gemini 3.1 Pro, non-thinking Gemini 3.1 Flash-Lite, non-thinking 0% 20% 40% 60% 80% 100% $0.001 $0.01 $0.10 $1.00 Cost per task, log scale Claude Fable 5, medium thinking

Source: Official Loss Run Extraction (Bevaya) leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Fable 5 medium thinking 85.3% $0.46
Claude Opus 5 medium thinking 85.3% $0.19
Gemini 3.6 Flash medium thinking 83.7% $0.15
Gemini 3.1 Pro non-thinking 83.5% $0.07
Claude Sonnet 5 medium thinking 83.4% $0.15
Claude Opus 4.8 medium thinking 83.2% $0.24
Claude Sonnet 4.6 non-thinking 82.9% $0.09
Claude Opus 4.8 non-thinking 82.9% $0.17
Gemini 3.6 Flash non-thinking 82.8% $0.05
Claude Sonnet 5 non-thinking 82.7% $0.21
Gemini 3.5 Flash medium thinking 82.6% $0.20
Gemini 3.1 Pro medium thinking 82.4% $0.12
GPT-5 non-thinking 82.1% $0.15
Gemini 3.5 Flash non-thinking 82% $0.05
GPT-4.1 non-thinking 80.4% $0.05
Gemini 3.1 Flash-Lite non-thinking 80.1% $0.008
Gemini 3.1 Flash-Lite medium thinking 79.9% $0.01
Claude Haiku 4.5 non-thinking 77.7% $0.03

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard Passed 85% in Aug 2026, by Claude Fable 5

Source: Official Loss Run Extraction (Bevaya) leaderboard

Claude Fable 5 on the official leaderboard passed 85% in June 2026

Models released before it came out already scored that high. All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Fable 5 Gemini 3.1 Pro GPT-5 GPT-4.1 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Fable 5 GPT-4.1

Source: Official Loss Run Extraction (Bevaya) leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Vision

All 636 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. Qwen 3.6 Plus 86%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MathVista In use Math reasoning in visual contexts (charts, geometry, puzzles) from 31 datasets. Answer accuracy. Seed2.1 Pro 90.7%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%

Data audit

All 9 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access The tasks are private.
  • Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
  • Pass Independent runs Scores from the official leaderboard (13 entries).
  • Pass Models scored 12 models we track, plus 1 other entry on its leaderboard.
  • Pass Newest score The newest is from July 2026, for Claude Opus 5.
  • Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11).
  • Note Headroom Saturated. The best is 85.3%, past 85% since June 2026.
Leaderboard import 3 pass
  • Pass Importer We read the board on bevaya.ai: 18 rows for models we track and 1 other entry at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check