University of Edinburgh
We track 6 benchmarks that University of Edinburgh made or helped make.
0 in use 5 with public code 4 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| MMLU-Redux Manually re-annotated subset of MMLU that fixes wrong labels. Multiple-choice accuracy. | Exam questions | Jun 2024 | | Fading |
| MMLongBench Vision-language models solve 13,331 long-context examples in five task types, such as visual RAG, needle search and many-shot in-context learning. Each example comes at five lengths from 8K to 128K tokens. | Finding facts in long inputs | May 2025 | | Fading |
| MIRB Models compare, analyze and reason across several images in four categories: perception, visual world knowledge, reasoning and multi-hop reasoning. | General visual questions | Jun 2024 | 37.7% | System cards only |
| OdysseyBench Agents recover requirements from conversation histories and complete workflows across documents, spreadsheets, PDFs, email, and calendars. Task-specific checks inspect the resulting file system and report the percentage of completed tasks. | Documents and spreadsheets | Aug 2025 | 58.7% | Not in cards |
| MMLU-Redux 2.0 Models answer 5,700 MMLU questions from all 57 subjects that experts re-annotated for errors. Flawed questions are marked so scoring can skip or fix them. | Exam questions | Aug 2024 | Not in cards | |
| XSum Models summarize a BBC news article in one sentence. ROUGE-1, ROUGE-2, and ROUGE-L compare the generated summary with the journalist's introductory summary. | Reading comprehension | Aug 2018 | Not in cards |