Benchmark Almanac

University College London

We track 8 benchmarks that University College London made or helped make.

0 in use 6 with public code 6 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Linguini Models solve 160 International Linguistic Olympiad problems with 894 questions in 75 mostly low-resource languages. All evidence needed is in the context. Reasoning in many languages Sep 2024 24.1% System cards only
LLM-WikiRace Models play the Wikipedia game: starting from one article, they follow one hyperlink at a time to reach a target article. At each step the model sees only the links on the current page, and a game is lost after 30 steps. Three difficulty levels are set by the shortest path between the two pages. Scores are the share of games won at each level. Puzzles and chess Feb 2026 23% Not in cards
AfroBench-Lite Models complete language understanding, question answering, reasoning, and generation tasks across 14 African languages. Seven datasets use task-specific metrics, including accuracy, F1, character-level translation scores, and exact match. Knowledge in many languages Jun 2025 77.7 Not in cards
LLM Uncertainty Bench Models answer questions across five language tasks. The original benchmark measures accuracy and uncertainty, while the port checks only the selected answer. Language reasoning Jan 2024 81.5% Not in cards
Continual Learning Exam Agents work through 120 professional tasks, each a ladder of five levels that must be passed in order. Later levels reuse files, tools and lessons from earlier ones. An evaluation agent grades against expert rubrics. Learning from the context Oct 2026 47.7% Not in cards
ContextBench Coding agents resolve 1,136 issues from 66 repositories in eight programming languages, and the benchmark checks which code they look at along the way. Each task has human-annotated gold context, and the tracker scores context recall, precision and efficiency over the agent's trajectory. It adds these process measures to the usual issue-resolution result. Fixing real repositories Feb 2026 60.6% Not in cards
SPA-Bench Agents use Android apps such as Airbnb, Spotify, and WeChat. The grader matches screenshots against task requirements and uses a vision model to resolve uncertain cases. Phones Oct 2024 1.35% Not in cards
PitVQA Models answer questions about a frame from pituitary surgery. Questions cover the surgical step, the instruments in view, and what the surgeon is doing. Medicine May 2024 Not in cards