Benchmark Almanac

AllenAI

We track 4 benchmarks that AllenAI made or helped make.

1 in use 2 with public code 4 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
IFBench 58 new out-of-domain verifiable output constraints. Scored by programmatic constraint checks (strict or loose). Following precise instructions Jul 2025 89.2% In use
DROP Reading comprehension requiring discrete operations (counting, sorting, arithmetic) over paragraphs. Scored by F1. Reading comprehension Mar 2019 93.5% Fading
HellaSwag Commonsense sentence completion with adversarially filtered wrong endings. Multiple-choice accuracy. Commonsense May 2019 97% Fading
WinoGrande 44K Winograd-style pronoun-resolution problems. Binary-choice accuracy. Commonsense Jul 2019 91.7% Fading