Benchmark Almanac

Benchmark Heaven

We track 4 benchmarks that Benchmark Heaven made or helped make.

0 in use 4 with public code 4 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
JevBench Decision models, such as TypeSafe AI's Jev and compatible systems, answer typed decision questions. The board scores accuracy, probability calibration, latency and cost. Other Sep 2026 98.4% Not in cards
AudioJevBench Models listen to a clip and assign probabilities to answers about speech or sound. The maker checks accuracy and calibration across 988 scored items. Audio understanding and voice Sep 2026 70.4 Not in cards
ImageJevBench v0.1.5 Models answer decisions about images and assign probabilities to the answers. The maker ranks capability using accuracy and calibration within cost and latency limits. General visual questions Oct 2026 16.1 Not in cards
ImageJevBench v0.3.0 Models make decisions about images and report probabilities. This version draws 300 public and 1,200 sealed items from a new pool. General visual questions Oct 2026 17.3 Not in cards