Benchmark Almanac

Adam Holter

We track 7 benchmarks that Adam Holter made or helped make.

0 in use 6 with public code 7 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
ButtonBench Models resist repeated prompts that tempt them to press a forbidden button. The benchmark scores refusal and the creativity of the resistance. Deception and alignment Dec 2025 97.6 Not in cards
AGI-Pilled Bench Models forecast when AI will reach milestones that already happened in 2026. The score is the median years after 2026 across the forecast tasks. Forecasting Oct 2026 6.50 Not in cards
HearingBench Models identify sounds from spectrogram images without hearing the audio. The test covers everyday sounds, instruments, vocals, words and sentences. Audio understanding and voice Oct 2026 46% Not in cards
RedstoneBench Agents design redstone circuits in Minecraft. A game server tests whether each circuit works and measures the space it occupies. Video games Nov 2026 88.7 Not in cards
ChinaBench Models answer politically sensitive questions about China. An independent judge labels each response compliant, refused or evasive. Deception and alignment Feb 2026 84.5% Not in cards
GarbleBench Models answer factual questions after controlled keyboard typos, dropped letters, stray characters and swaps corrupt the text. Instructions and chat Oct 2026 55.2% Not in cards
ClutterBench Agents turn an article into an HTML graphic. A model grades the share of the drawn area occupied by unnecessary or oversized elements. Web and app front ends Oct 2026 28.8% Not in cards