Benchmark Almanac

UCLA

We track 10 benchmarks that UCLA made or helped make.

2 in use 8 with public code 3 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
MathVista Math reasoning in visual contexts (charts, geometry, puzzles) from 31 datasets. Answer accuracy. Visual math and diagrams Oct 2023 90.7% In use
LogicVista Models answer 448 multiple-choice questions on spatial, deductive, inductive, numeric, and mechanical reasoning set in images. Scored by accuracy. Visual math and diagrams Jul 2024 91.9% In use
WinoBias Models resolve pronouns linked to occupations in type 1 and type 2 sentences. Accuracy gaps show gender bias in coreference. Bias, fairness and politics Apr 2018 System cards only
ScienceQA Models answer about 21,000 multiple-choice science questions from school curricula, many with images. Scored by accuracy. General visual questions Sep 2022 97.5% Dropped
Inter-GPS (Geometry3K) Models solve high school geometry problems that combine a diagram and text from the Geometry3K test set. Scored by accuracy. Visual math and diagrams May 2021 48.6% Dropped
VideoPhy-2 200 action-centric prompts. Generated videos human-rated for semantic adherence, physical commonsense and physical-rule grounding. Physical prediction Mar 2025 24.9% System cards only
ExPUNations Models explain English puns and write new ones from keywords. Human raters check whether a generated pun supports both meanings. Creative writing Oct 2022 Not in cards
GenEval 2 Models generate images from compositional prompts. A visual question answering model checks whether each requested object, attribute, count, and relation appears. Image generation and editing Dec 2025 Not in cards
On Evaluating User Simulators for Interactive Agents Models play the user in replays of 351 real sessions. The evaluation compares simulated behavior with observed behavior and measures agreement with human rankings of nine agents. Multi-turn conversation Nov 2026 Not in cards
VideoPhy Text-to-video models generate clips from captions of physical interactions between solids and fluids. Human raters judge caption match and physical commonsense. Physical prediction Jun 2024 Not in cards