Benchmark Almanac

the AHELM team

We track 3 benchmarks that the AHELM team made or helped make.

0 in use 1 with public code 2 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
COREBench (Conversational Reasoning) Models listen to a conversation with several speakers and answer a question that can require information from multiple turns. Audio understanding and voice Jun 2025 81.3% Not in cards
PARADE Audio Models answer questions about occupations and social status using audio clips. The evaluation tests whether voice cues bias their answers. Bias, fairness and politics Mar 2025 63.4% Not in cards
Audio PAIRS Models answer spoken questions adapted from PAIRS. The evaluation checks whether changing demographic cues changes their choices. Bias, fairness and politics Nov 2024 Not in cards