Benchmark Almanac

Queen Mary University of London

We track 5 benchmarks that Queen Mary University of London made or helped make.

0 in use 4 with public code 2 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
PianoBench Models answer questions about piano performances from recordings or symbolic transcriptions. The authors compare accuracy with and without access to the music. Audio understanding and voice Nov 2026 93.2% Not in cards
CMI-RewardBench Models judge musical quality and how well recordings follow requests. Some requests combine text, lyrics, and reference audio. Scores measure agreement with human listeners. Audio understanding and voice Feb 2026 0.60 Not in cards
CMI-Bench Models listen to music and answer requests such as naming an instrument, marking beats, or transcribing lyrics. Each task has its own scoring rule. Audio understanding and voice Jun 2025 37.9 Not in cards
ATEPP Models identify pianists from the notes, timing, and dynamics of their performances. The original evaluation includes a mixed repertoire and three composer-specific tests. Audio understanding and voice May 2022 Not in cards
MUSDB-ALT Models transcribe 39 English songs from the MUSDB18 test set. Line timings and separate vocal recordings let researchers compare transcription with and without the accompaniment. Speech recognition and translation Apr 2025 Not in cards