Benchmark Almanac

University of Cambridge

We track 6 benchmarks that University of Cambridge made or helped make.

1 in use 5 with public code 2 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
ZeroBench 100 hand-made visual questions that all frontier models originally failed. Scored by exact answer, pass@1 and pass@5. General visual questions Feb 2025 54.6% In use
ProteinLMBench Models answer 944 hand-checked multiple choice questions on protein sequence, structure and function. Genomics and life sciences Jun 2024 57.9% Dropped
RivaBench Models answer questions about stand-up routines and academic talks using speech and visual cues. They also classify clips as real or synthetic. Video understanding Feb 2025 Not in cards
VSR Models judge whether captions with 65 kinds of spatial relation, such as under or facing, are true of an image. Over 10,000 text and image pairs. Spatial reasoning and counting Apr 2022 Not in cards
VSR Random Split Models judge whether image captions correctly describe spatial relations on the randomly partitioned test set. The score is accuracy over 2,195 test examples. Spatial reasoning and counting Apr 2022 Not in cards
VSR Zero-Shot Split Models judge spatial captions on a test set whose object concepts do not appear in training. The score is accuracy over 616 test examples. Spatial reasoning and counting Apr 2022 Not in cards