Benchmark Almanac

Max Planck Institute for Informatics

We track 4 benchmarks that Max Planck Institute for Informatics made or helped make.

0 in use 1 with public code 1 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
TACoS Models find the time span in a cooking video that matches a sentence. Labs score temporal grounding by mean IoU with the annotated span. Video understanding Sep 2014 49.6% Dropped
KRIS-Bench Image editing models follow 1,267 instructions that need factual, conceptual or procedural knowledge across 22 tasks. A GPT-4o judge scores the edits. Image generation and editing May 2025 66.2 System cards only
Ref-DAVIS17 Models segment and track the object that a sentence describes through each frame of a DAVIS 2017 video. Scored by J&F, the mean of mask overlap and contour accuracy. Video understanding Mar 2018 Not in cards
Saarland MIDI-Audio Piano Music Models transcribe student piano performances. The Disklavier captured the reference notes as the musicians played. Audio understanding and voice Oct 2011 Not in cards