Benchmark Almanac

University of Wisconsin-Madison

We track 9 benchmarks that University of Wisconsin-Madison made or helped make. Its models are on its lab page.

0 in use 6 with public code 6 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
SpatialEval Maze-Nav Models answer questions about routes and turns in mazes given as text or images. 1,500 multiple-choice questions scored by accuracy. Spatial reasoning and counting Jun 2024 41% System cards only
SpatialEval Spatial-Map Models answer questions about relations and counts among objects placed on a map given as text or images. 1,500 questions scored by accuracy. Spatial reasoning and counting Jun 2024 64.3% System cards only
Orak LLM agents play 12 popular video games across most genres through an MCP interface. Each game scores in-game progress or outcomes, and the board ranks models by their average rank across the games. Video games Jun 2025 3.90 Fading
SlopCodeBench Agents repeatedly extend their own code across 36 problems and 196 checkpoints as the specification changes. Tests check correctness, and metrics track structural erosion and verbosity. Building whole projects Mar 2026 32.1% Not in cards
SimBench (PyChrono) Models write and revise PyChrono simulation programs over three turns. A model grades each program using the reference solution and API documentation. Scientific code Aug 2024 49.0 Not in cards
HarnessAudit-Bench Agents run inside execution harnesses such as Claude Code, Codex and OpenClaw on 210 tasks across eight domains, from finance to software engineering, with safety rules built in. The audit reads the full trajectory through channels the agent cannot change. It scores whether the agent stays inside its permission and information-flow policies, completes checkpoints, and keeps both under prompt injection, vague goals and tool errors. Refusing harmful requests May 2026 0.41 Not in cards
Humor in AI: Cartoon Captioning Models write captions for New Yorker cartoons. Human judges and language models compare the captions with contest entries from different ranking groups. Creative writing Jun 2024 73.9% Not in cards
SAC-Bench Models identify sounds and their order when a spoken claim contradicts the recording. Matched clips test whether answers follow the acoustic evidence. Audio understanding and voice Oct 2026 91.3% Not in cards
LLaVA-Bench (In-the-Wild) Multimodal assistants answer open questions about a small set of diverse real images. A GPT-4 judge scores each answer relative to a reference answer. General visual questions Apr 2023 Not in cards