Benchmark Almanac

Universitat Pompeu Fabra

We track 8 benchmarks that Universitat Pompeu Fabra made or helped make.

1 in use 7 with public code 0 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
HumMusQA Models answer 320 hand-written questions about music, curated and checked by people with musical training. Audio understanding and voice Mar 2026 75.8% In use
Can I Play It? Models assign a difficulty grade from one to nine to each piano score. The original study reports balanced accuracy across the nine grades. Audio understanding and voice May 2022 Not in cards
CatCoLA Models classify Catalan sentences by whether a native speaker would consider them grammatical. Reading in many languages May 2024 Not in cards
EsCoLA Models judge the grammatical acceptability of Spanish sentences taken from linguistic publications. Reading in many languages May 2024 Not in cards
Hidden Voices Audio Difficulty Models rank piano works by Black women composers from recordings. The reference ordering comes from the Hidden Voices project’s seven difficulty grades. Audio understanding and voice Mar 2024 Not in cards
Lakh MIDI MASD Style Classification Models identify which of 25 styles apply to a MIDI file. The released mapping labels 17,785 files, and each file can have multiple labels. Audio understanding and voice Sep 2017 Not in cards
Lakh MIDI Top-MAGD Genre Classification Models identify which of 13 genres apply to a MIDI file. The released mapping labels 22,535 files, and each file can have multiple labels. Audio understanding and voice Sep 2017 Not in cards
MSD-I Multimodal Genre Classification Models predict a track’s genre from its audio and album cover. The published test contains 4,649 tracks across 15 genres. Audio understanding and voice Apr 2018 Not in cards