Benchmark Almanac

Tel Aviv University

We track 8 benchmarks that Tel Aviv University made or helped make.

0 in use 8 with public code 4 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
CommonSenseQA Models answer five-way multiple-choice questions built from ConceptNet relations that need everyday background knowledge. Scored by accuracy. Commonsense Nov 2018 90.4% Dropped
ZeroSCROLLS/QuALITY Models answer QuALITY multiple-choice questions about long stories zero-shot, as one task of the ZeroSCROLLS suite. Scoring is accuracy. Reasoning over long documents May 2023 95.2% Dropped
CommonsenseQA 2.0 Models answer 14,343 yes or no commonsense questions that players wrote to fool an AI in a game. Scored by accuracy. Commonsense Jan 2022 67.8% Not in cards
CryptanalysisBench Agents attack 191 real cryptographic primitives, mostly from four NIST standardization competitions. Tier 1 has known practical attacks, and Tier 2 has none, so a solve there is a new break. Finding and exploiting real bugs Jul 2026 34.2% Not in cards
AssistantBench Web agents do 214 realistic, time-consuming tasks on the open web, such as monitoring real-estate markets. Answers are checked automatically against gold answers. Web browsing Jul 2024 Not in cards
ParaShoot Models answer Hebrew questions by selecting words from a Hebrew Wikipedia passage. Scorers compare the selected text with human answers. Reading in many languages Sep 2021 Not in cards
QAMPARI Models return multiple answers to a question using evidence scattered across Wikipedia passages. Each question has at least five correct answers. Questions that need search May 2022 Not in cards
SCROLLS Models do seven long-text tasks such as summarization, question answering and natural language inference over naturally long documents. Reasoning over long documents Jan 2022 Not in cards