Benchmark Almanac

University of Amsterdam

We track 8 benchmarks that University of Amsterdam made or helped make.

0 in use 6 with public code 1 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
LAMBADA Models predict the last word of narrative passages where humans can guess it only from the whole passage. Scored by accuracy. Reading comprehension Jun 2016 86.9% Dropped
SIFo Models follow a chain of dependent instructions, where each step uses the output of the one before. Only the final answer is checked, across 800 samples in four tasks. Following precise instructions Jun 2024 77.3% Dropped
CLEF 2025 SimpleText Task 1 Models simplify individual sentences or whole Cochrane abstracts. The organizers tested new reviews and scored both tasks against plain English references. Medicine Nov 2024 43.0 Not in cards
REALM High-fidelity simulation benchmark for VLA generalization: 15 perturbation factors, 7 skills, 3,500+ objects. Success rate validated against real-robot results. Simulated robots Dec 2025 53.1% Not in cards
CLEF 2026 SimpleText Task 2 Models detect or classify unsupported additions to simplified Cochrane abstracts. A generation subtask compares rewrites made with and without explicit checks against the source. Faithful to sources Sep 2025 0.34 Not in cards
CLEF 2025 SimpleText Task 2 Models flag unsupported sentences, classify distortions in simplified scientific text, or rewrite Cochrane abstracts while avoiding new claims. Faithful to sources Nov 2024 Not in cards
CLEF 2026 SimpleText Task 1 Models rewrite sentences or whole Cochrane abstracts for the public. The organizers compared outputs with plain-language summaries from 175 new systematic reviews. Medicine Sep 2025 Not in cards
Cochrane-auto Models simplify sentences, paragraphs, or abstracts from Cochrane reviews. The authors aligned technical text with plain English summaries. Medicine Nov 2024 Not in cards