Loss Run Extraction (Bevaya)
Models extract claims and policy tables from insurance loss reports into JSON. The main metric counts documents where every field matches the reference.
Bevaya compares its model with 12 frontier models on 346 documents. The dashboard also reports field accuracy, latency, and cost.
- Made by
- Bevaya Labs
- Released
- August 2026
- Kind
- Documents, charts and text in images
- Level
- University
- Scored
- Checked automatically
- Answers
- Open-ended
- Tasks
- Kept private
- Measures
- Capability
- Subject
- Economics and business
- Language
- English
- Interaction
- Single turn
- Items from
- Real work and use
- Refresh
- One fixed set
- Run by
- A host
- In views
- Law and finance
- Strongest model tested
- Claude Opus 5 #11 on the capability index
- Last model added
- July 2026
- Scores refreshed
- November 20th, 2026
Our Loss Run Model tops the leaderboard with 93.1% Claude Fable 5 leads with 85.3% among models released by July 2026
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Claude Fable 5, at 85.3%. This view shows only models we track, each at its best.
The newest model with a score here came out in July 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6 Astra
- and 14 more
- 1 Our Loss Run Model serial 93.1%
- 2 Claude Fable 5 medium thinking 85.3%
- 3 Claude Opus 5 medium thinking 85.3%
- 4
Gemini 3.6 Flash medium thinking 83.7%
- 5
Gemini 3.1 Pro non-thinking 83.5%
- 6 Claude Sonnet 5 medium thinking 83.4%
- 7 Claude Opus 4.8 medium thinking 83.2%
- 8 Claude Sonnet 4.6 non-thinking 82.9%
- 9
Gemini 3.5 Flash medium thinking 82.6%
- 10 GPT-5 non-thinking 82.1%
- 11 GPT-4.1 non-thinking 80.4%
- 12
Gemini 3.1 Flash-Lite non-thinking 80.1%
- 13 Claude Haiku 4.5 non-thinking 77.7%
Source: Official Loss Run Extraction (Bevaya) leaderboard
- 1 Claude Fable 5 medium thinking 85.3%
- 2 Claude Opus 5 medium thinking 85.3%
- 3
Gemini 3.6 Flash medium thinking 83.7%
- 4
Gemini 3.1 Pro non-thinking 83.5%
- 5 Claude Sonnet 5 medium thinking 83.4%
- 6 Claude Opus 4.8 medium thinking 83.2%
- 7 Claude Sonnet 4.6 non-thinking 82.9%
- 8
Gemini 3.5 Flash medium thinking 82.6%
- 9 GPT-5 non-thinking 82.1%
- 10 GPT-4.1 non-thinking 80.4%
- 11
Gemini 3.1 Flash-Lite non-thinking 80.1%
- 12 Claude Haiku 4.5 non-thinking 77.7%
Source: Official Loss Run Extraction (Bevaya) leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Our Loss Run Model | 93.1% | serial | Official leaderboard | |
| 2 | Claude Fable 5 | 85.3% | medium thinking | Jun 2026 | Official leaderboard |
| 3 | Claude Opus 5 | 85.3% | medium thinking | Jul 2026 | Official leaderboard |
| 4 | | 83.7% | medium thinking | Jul 2026 | Official leaderboard |
| 5 | | 83.5% | non-thinking | Feb 2026 | Official leaderboard |
| 6 | Claude Sonnet 5 | 83.4% | medium thinking | Jun 2026 | Official leaderboard |
| 7 | Claude Opus 4.8 | 83.2% | medium thinking | May 2026 | Official leaderboard |
| 8 | Claude Sonnet 4.6 | 82.9% | non-thinking | Feb 2026 | Official leaderboard |
| 9 | | 82.6% | medium thinking | May 2026 | Official leaderboard |
| 10 | GPT-5 | 82.1% | non-thinking | Aug 2025 | Official leaderboard |
| 11 | GPT-4.1 | 80.4% | non-thinking | Apr 2025 | Official leaderboard |
| 12 | | 80.1% | non-thinking | Mar 2026 | Official leaderboard |
| 13 | Claude Haiku 4.5 | 77.7% | non-thinking | Oct 2025 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Claude Fable 5 | 85.3% | medium thinking | Jun 2026 | Official leaderboard |
| 2 | Claude Opus 5 | 85.3% | medium thinking | Jul 2026 | Official leaderboard |
| 3 | | 83.7% | medium thinking | Jul 2026 | Official leaderboard |
| 4 | | 83.5% | non-thinking | Feb 2026 | Official leaderboard |
| 5 | Claude Sonnet 5 | 83.4% | medium thinking | Jun 2026 | Official leaderboard |
| 6 | Claude Opus 4.8 | 83.2% | medium thinking | May 2026 | Official leaderboard |
| 7 | Claude Sonnet 4.6 | 82.9% | non-thinking | Feb 2026 | Official leaderboard |
| 8 | | 82.6% | medium thinking | May 2026 | Official leaderboard |
| 9 | GPT-5 | 82.1% | non-thinking | Aug 2025 | Official leaderboard |
| 10 | GPT-4.1 | 80.4% | non-thinking | Apr 2025 | Official leaderboard |
| 11 | | 80.1% | non-thinking | Mar 2026 | Official leaderboard |
| 12 | Claude Haiku 4.5 | 77.7% | non-thinking | Oct 2025 | Official leaderboard |
Rankings by part
We read 2 parts from the benchmark’s own leaderboard: Claims and Policy. Each ranks models on one part of the test. Claude Opus 5 leads Claims and Claude Fable 5 leads Policy. Arrows show how far a model moved from its overall rank among the models we track.
Claims
12 models we track have a score here.
- 1 Claude Opus 5 ↑1 87.3%
- 2 Claude Fable 5 ↓1 85.6%
- 3 Claude Opus 4.8 ↑3 85.3%
- 4 Claude Sonnet 4.6 ↑3 84.9%
- 5
Gemini 3.1 Pro ↓1 84.8%
- 6
Gemini 3.6 Flash ↓3 84.8%
- 7 Claude Sonnet 5 ↓2 84.7%
- 8
Gemini 3.5 Flash 83.1%
- 9
Gemini 3.1 Flash-Lite ↑2 83.1%
- 10 GPT-5 ↓1 82.8%
Policy
12 models we track have a score here.
- 1 Claude Fable 5 85.1%
- 2 Claude Opus 5 83.8%
- 3
Gemini 3.6 Flash 82.9%
- 4
Gemini 3.1 Pro 82.6%
- 5 Claude Sonnet 5 82.4%
- 6 Claude Opus 4.8 82.2%
- 7
Gemini 3.5 Flash ↑1 82.2%
- 8 GPT-5 ↑1 81.6%
- 9 Claude Sonnet 4.6 ↓2 81.5%
- 10 GPT-4.1 79.8%
The best score, 85.3%, costs $0.46 a task
Runs on the cost frontier score higher than any run that's cheaper. GPT-4.1 at non-thinking comes within 4.9 points of the best for $0.05 a task, 10 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.
Source: Official Loss Run Extraction (Bevaya) leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| Claude Fable 5 | medium thinking | 85.3% | $0.46 | ||
| Claude Opus 5 | medium thinking | 85.3% | $0.19 | ||
| medium thinking | 83.7% | $0.15 | |||
| non-thinking | 83.5% | $0.07 | |||
| Claude Sonnet 5 | medium thinking | 83.4% | $0.15 | ||
| Claude Opus 4.8 | medium thinking | 83.2% | $0.24 | ||
| Claude Sonnet 4.6 | non-thinking | 82.9% | $0.09 | ||
| Claude Opus 4.8 | non-thinking | 82.9% | $0.17 | ||
| non-thinking | 82.8% | $0.05 | |||
| Claude Sonnet 5 | non-thinking | 82.7% | $0.21 | ||
| medium thinking | 82.6% | $0.20 | |||
| medium thinking | 82.4% | $0.12 | |||
| GPT-5 | non-thinking | 82.1% | $0.15 | ||
| non-thinking | 82% | $0.05 | |||
| GPT-4.1 | non-thinking | 80.4% | $0.05 | ||
| non-thinking | 80.1% | $0.008 | |||
| medium thinking | 79.9% | $0.01 | |||
| Claude Haiku 4.5 | non-thinking | 77.7% | $0.03 |
The best score, 85.3%, costs $0.46 a task
Runs on the cost frontier score higher than any run that's cheaper. GPT-4.1 at non-thinking comes within 4.9 points of the best for $0.05 a task, 10 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.
Source: Official Loss Run Extraction (Bevaya) leaderboard
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| Claude Fable 5 | medium thinking | 85.3% | $0.46 | ||
| Claude Opus 5 | medium thinking | 85.3% | $0.19 | ||
| medium thinking | 83.7% | $0.15 | |||
| non-thinking | 83.5% | $0.07 | |||
| Claude Sonnet 5 | medium thinking | 83.4% | $0.15 | ||
| Claude Opus 4.8 | medium thinking | 83.2% | $0.24 | ||
| Claude Sonnet 4.6 | non-thinking | 82.9% | $0.09 | ||
| Claude Opus 4.8 | non-thinking | 82.9% | $0.17 | ||
| non-thinking | 82.8% | $0.05 | |||
| Claude Sonnet 5 | non-thinking | 82.7% | $0.21 | ||
| medium thinking | 82.6% | $0.20 | |||
| medium thinking | 82.4% | $0.12 | |||
| GPT-5 | non-thinking | 82.1% | $0.15 | ||
| non-thinking | 82% | $0.05 | |||
| GPT-4.1 | non-thinking | 80.4% | $0.05 | ||
| non-thinking | 80.1% | $0.008 | |||
| medium thinking | 79.9% | $0.01 | |||
| Claude Haiku 4.5 | non-thinking | 77.7% | $0.03 |
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official Loss Run Extraction (Bevaya) leaderboard
Claude Fable 5 on the official leaderboard passed 85% in June 2026
Models released before it came out already scored that high. All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official Loss Run Extraction (Bevaya) leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Vision
- MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy.
Qwen 3.6 Plus 86%
- MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
- CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
- MathVista In use Math reasoning in visual contexts (charts, geometry, puzzles) from 31 datasets. Answer accuracy.
Seed2.1 Pro 90.7%
- MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy.
Qwen 3.8 Max 97.7%
- MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy.
Qwen 3.8 Max 85.9%
Data audit
All 9 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access The tasks are private.
- Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
- Pass Independent runs Scores from the official leaderboard (13 entries).
- Pass Models scored 12 models we track, plus 1 other entry on its leaderboard.
- Pass Newest score The newest is from July 2026, for Claude Opus 5.
- Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is Claude Opus 5 (#11).
- Note Headroom Saturated. The best is 85.3%, past 85% since June 2026.
Leaderboard import 3 pass
- Pass Importer We read the board on bevaya.ai: 18 rows for models we track and 1 other entry at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.