Benchmark Almanac

ClaimBench

d1-ultra 100% tops its leaderboard
Not in model cards

Models label whether a 10-K excerpt supports, contradicts or says nothing about a claim, across 1,960 claims. Small decision models are scored on accuracy, cost and latency.

Built to test how much claim checking cheap decision models can take over within a 1% error budget. The board has 20 decision models.

Made by
Vals AI
Released
October 2026
Kind
Finance
Level
University
Scored
Checked automatically
Answers
Multiple choice
20
Entries on its own leaderboard, 0 models we track

d1-ultra tops the leaderboard with 100%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

20 entries. Scroll sideways for the rest.

  1. 100.0 d1-ultra
  2. 99.7 Decider V1.2
  3. 99.2 Drex Max
  4. 99.1 Mercury Decide
  5. 98.8 Decider V1.1 27B
  6. 98.7 d1
  7. 98.2 Microsoft-Decision-1
  8. 98.1 Jev 1.14
  9. 98.0 Drex v1.5
  10. 97.7 Jev 1.13.0
  11. 97.3 Drex v2.0
  12. 96.9 GPT-6 Luna Decisions
  13. 96.5 Decider Flash V1.2
  14. 95.1 Clef
  15. 94.4 Solar Decide
  16. 89.8 Tev1 4B Experimental
  17. 87.7 Clef Omni
  18. 85.0 Classify-1
  19. 84.8 Clef Flash
  20. 82.5 Solar Decide Flash
  1. 1 d1-ultra 100%
  2. 2 Decider V1.2 99.7%
  3. 3 Drex Max 99.2%
  4. 4 Mercury Decide 99.1%
  5. 5 Decider V1.1 27B 98.8%
  6. 6 d1 98.7%
  7. 7 Microsoft-Decision-1 98.2%
  8. 8 Jev 1.14 98.1%
  9. 9 Drex v1.5 98%
  10. 10 Jev 1.13.0 97.7%
  11. 11 Drex v2.0 97.3%
  12. 12 GPT-6 Luna Decisions 96.9%
  13. 13 Decider Flash V1.2 96.5%
  14. 14 Clef 95.1%
  15. 15 Solar Decide 94.4%
  16. 16 Tev1 4B Experimental 89.8%
  17. 17 Clef Omni 87.7%
  18. 18 Classify-1 85%
  19. 19 Clef Flash 84.8%
  20. 20 Solar Decide Flash 82.5%

Source: Official ClaimBench leaderboard

#EntryScoreSettingDateSource
1 d1-ultra 100% Official leaderboard
2 Decider V1.2 99.7% Official leaderboard
3 Drex Max 99.2% Official leaderboard
4 Mercury Decide 99.1% Official leaderboard
5 Decider V1.1 27B 98.8% Official leaderboard
6 d1 98.7% Official leaderboard
7 Microsoft-Decision-1 98.2% Official leaderboard
8 Jev 1.14 98.1% Official leaderboard
9 Drex v1.5 98% Official leaderboard
10 Jev 1.13.0 97.7% Official leaderboard
11 Drex v2.0 97.3% Official leaderboard
12 GPT-6 Luna Decisions 96.9% Official leaderboard
13 Decider Flash V1.2 96.5% Official leaderboard
14 Clef 95.1% Official leaderboard
15 Solar Decide 94.4% Official leaderboard
16 Tev1 4B Experimental 89.8% Official leaderboard
17 Clef Omni 87.7% Official leaderboard
18 Classify-1 85% Official leaderboard
19 Clef Flash 84.8% Official leaderboard
20 Solar Decide Flash 82.5% Official leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Law, finance and medicine

All 179 in Law, finance and medicine
  • MedXpertQA-MM In use Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy. Qwen 3.8 Max 80.4%
  • HealthBench Professional In use 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. Claude Sonnet 5.5 69.2%
  • HealthBench In use 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. GPT-5 67.2%
  • FinSearchComp In use Agents search the web to fetch live financial data, look up history, and run multi-step investigations, across 635 expert-written questions. Scored by answer accuracy. Ling-3.1-flash 81.7%
  • KBL In use Models answer Korean legal knowledge and reasoning tasks and Korean bar exam questions. It has 7 knowledge tasks, 4 reasoning tasks and 2,510 bar exam items. Solar Pro 4 77.5%
  • KorMedMCQA In use Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. Solar Pro 4 93.2%