Benchmark Almanac

University of Michigan

We track 9 benchmarks that University of Michigan made or helped make.

0 in use 6 with public code 4 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
YouCook2 Models describe the procedure steps in YouTube cooking videos. Scored by CIDEr against human step captions. Video captioning Mar 2017 188.3 Dropped
CoreCognition Models answer true or false and multiple choice questions on 12 core knowledge concepts from developmental cognitive science, such as object permanence and perspective taking. General visual questions Oct 2024 74.9% System cards only
ExpertLongBench Models write documents for 11 professional tasks. A judge compares facts in each document with a reference checklist and scores precision and recall. Tasks from real jobs Jun 2025 33.4% Not in cards
MMTU Models answer about 30,000 questions over real tables across 25 tasks, such as data cleaning, table joins, column and table transformation, table question answering and turning language into code. The tasks come from decades of computer science research on tabular data. Answers are checked automatically against references. Data science and SQL Jun 2025 69.6% Not in cards
MobileAgentBench Agents complete 100 tasks across ten Android apps. The grader checks app state and accessibility events, so agents can take different paths. Phones Jun 2024 8.00% Not in cards
EPIC-KITCHENS VISOR Hand Object Segmentation Models outline each hand and the object it touches in a kitchen image. They also identify which hand appears and whether it touches anything. Detection and captioning Aug 2022 Not in cards
EPIC-KITCHENS VISOR Semi-Supervised Video Object Segmentation Models receive masks for hands and objects in the first frame of a kitchen video. They track those objects through later frames, including after occlusion. Video understanding Aug 2022 Not in cards
GovReport Models summarize long US government reports from the Government Accountability Office and the Congressional Research Service. The reports are much longer than earlier summarization sets, and each has an expert-written summary. Scores are ROUGE overlap with the reference summary. Reasoning over long documents Apr 2021 Not in cards
YouCook2 Dense Video Description Models locate cooking steps and describe each step. The score is CIDEr for the complete dense video description task. Video captioning Mar 2017 Not in cards