Benchmark Almanac

AIRI

We track 2 benchmarks that AIRI made or helped make.

0 in use 2 with public code 1 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
BABILong Models answer bAbI reasoning questions whose supporting facts are scattered through very long distractor text, at lengths up to millions of tokens. Reasoning over long documents Jun 2024 72.4% Not in cards
SPY Models locate personal information about the author of a synthetic medical consultation or legal question. Other people mentioned in the text fall outside the annotation policy. Following precise instructions Sep 2024 Not in cards