Typed Decision Bench
Models assign probabilities to decisions about a given state. The benchmark scores accuracy and calibration across five suites.
Blobfish publishes run totals and verifier results.
- Made by
- Blobfish AI
- Released
- September 2026
- Kind
- Running a business
- Level
- Expert
- Scored
- Checked automatically
- Answers
- Prediction
- Tasks
- All public
- Measures
- Capability
- Language
- English
- Interaction
- Single turn
- Items from
- Another benchmark, Generated
- Refresh
- New versions replace it
- Run by
- Anyone
- In views
- Law and finance
jev-1.13.0 tops the leaderboard with 81.7
Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.
- 1 jev-1.13.0 81.7
- 2 jevfish-v4-r4b 78.2
- 3 decider-2b-v8 77.6
- 4 s1-scorer-4b-v2b 77.2
- 5 openjev-latest 75.6
- 6 nanojev-0.6b 56.8
Source: Official Typed Decision Bench leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | jev-1.13.0 | 81.7 | jevfish_bench runner · hosted API | Sep 2026 | Official leaderboard |
| 2 | jevfish-v4-r4b | 78.2 | jevfish_bench runner · jevfish_server (HF transformers) · RunPod A40 | Sep 2026 | Official leaderboard |
| 3 | decider-2b-v8 | 77.6 | jevfish_bench runner · jevfish_bench.competitors.decider_server · local MPS | Sep 2026 | Official leaderboard |
| 4 | s1-scorer-4b-v2b | 77.2 | jevfish_bench runner · jevfish_bench.competitors.scorer_server · RunPod eval pod | Sep 2026 | Official leaderboard |
| 5 | openjev-latest | 75.6 | jevfish_bench runner · hosted API | Sep 2026 | Official leaderboard |
| 6 | nanojev-0.6b | 56.8 | jevfish_bench runner · jevfish_bench.competitors.nanojev_server · RunPod eval pod | Sep 2026 | Official leaderboard |
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Economic work
- JobBench In use 130 tasks from 35 occupations workers most want delegated. Agent works through messy reference files, graded by fact-anchored binary rubrics. GLM-5.3 61.4%
- GDPval In use Real work tasks from 44 occupations with reference files. Experts blindly compare model deliverables with expert work. Metric is win-or-tie rate.
Seed2.1 Pro 87.9%
- GDPval-AA v2.1 In use Agents with shell and web access complete 220 GDPval gold tasks from 44 occupations. Blind pairwise comparisons give Elo ratings. Claude Opus 5.5 1,846
- AA-Briefcase v1.1 In use Version 1.1 of Artificial Analysis's multi-hour office work evaluation, with Elo ratings fitted by Crowd-BT. Claude Opus 5.5 1,822
- Agents' Last Exam v1.1 In use Agents complete 122 public long-horizon professional workflow tasks across 55 sub-industries. Deterministic graders check outcomes against hidden references. GPT-6.2 Astra 47.5%
- CoWorkBench In use Agent completes long-horizon productivity tasks across computer science, finance, law and medicine. Qwen in-house scoring (method unpublished). Claude Fable 5 75.9%
Data audit
All 8 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 2 pass 3 notes
- Pass Independent runs Scores from the official leaderboard (6 entries).
- Pass Models scored 6 entries on its leaderboard, none a model we track.
- Note Newest score No scores yet.
- Note Frontier models No models scored yet.
- Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
- Pass Importer We read the board on blobfish.ai: 6 entries at the last read, none a model we track.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.