Benchmark Almanac

Northwestern University

We track 6 benchmarks that Northwestern University made or helped make.

0 in use 5 with public code 5 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
MindCube Models answer 21,154 multiple choice questions about scenes seen from a few views. Questions test positions, orientations, and what-if movements. Spatial reasoning and counting Jun 2025 47.6% Fading
MM-FinEval Models forecast financial outcomes from earnings call transcripts, slides, and audio. The authors compare 19 models using directional accuracy and absolute error. Finance Sep 2026 88% Not in cards
Realtime-Bench Agents act in 50 live game and web tasks where a late action fails: 30 web tasks (travel, shopping, services and security) and 20 games. The environment keeps moving while the model thinks. Three clocks separate speed from skill: Wall Clock runs on real time, Paused Clock stops while the model thinks, and Token Clock advances with each generated token. Scores are task success rates over three runs. Web browsing Oct 2026 42.7% Not in cards
Embodied First Exam Coding agents answer frozen robot tasks from many benchmark suites with a program. Each program is verified in a separate sandbox, run on Harbor. Simulated robots Oct 2026 49.6% Not in cards
HumanCLAW-Bench Models control a simulated human body from egocentric images to find objects, navigate, and sit. The benchmark has 1,218 episodes across 41 houses. Success requires geometric completion and the agent's acknowledgment. Simulated robots Jul 2026 11.1% Not in cards
VIPBench Models predict whether listeners hear the same voice in two recordings. The authors compare eleven speech models against judgments from 1,290 listeners. Speech recognition and translation Nov 2026 Not in cards