Benchmark Almanac

University of Southern California

We track 8 benchmarks and 2 arenas that University of Southern California made or helped make.

1 in use 7 with public code 7 with a leaderboard here

Arena leaderboards

Top five in each of its 2 arenas with three or more rated models, the most rated first.

Benchmarks

Name Kind Released Best score Status
Charades-STA Models read a sentence describing an action and return its start and end time in a Charades indoor video. Scored by temporal IoU with the labeled moment. Video understanding May 2017 64% Fading
COPA Models choose which of two alternatives is the more plausible cause or effect of a premise, across 1,000 questions. Scored by accuracy. Commonsense Mar 2011 96% Dropped
DeputyBench Agents handle business requests under paired permission settings. The evaluator checks whether they act when authorized and seek evidence or refrain when authorization is missing. Simulated companies Nov 2026 92.6% Not in cards
CounselBench-Adv Models answer 120 questions written by mental health professionals. Each question targets one of six problems identified in earlier counseling responses. Wellbeing and sycophancy May 2025 0.37 Not in cards
CounselBench-Eval Models answer 100 questions that people posted for therapists. Mental health professionals rated responses on six dimensions. Medicine May 2025 5 Not in cards
ActiveVision Models inspect images repeatedly to count regions, trace structures, and compare visual attributes. The released split has five examples for each of 17 tasks. Exact match checks the final answer after normalizing case, spacing, and separators. Spatial reasoning and counting Jul 2026 50.6% Not in cards
ExPUNations Models explain English puns and write new ones from keywords. Human raters check whether a generated pun supports both meanings. Creative writing Oct 2022 Not in cards
IEMOCAP Models identify emotions in recorded dialogues. NVIDIA evaluates this task within AIR-Bench. Audio understanding and voice Nov 2008 Not in cards