Benchmark Almanac

University of California

We track 16 benchmarks that University of California made or helped make.

0 in use 15 with public code 5 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
GeoGuesser Bench Agents explore offline Street View panoramas to report coordinates, name places, and find objects. Coordinate guesses earn distance points. Other answers require an exact match. Spatial reasoning and counting Nov 2026 22.8 Not in cards
3DSceneBench Agents reconstruct interactive MuJoCo scenes from images and task descriptions. Verifiers measure appearance, geometry, and physical functionality, then average the three scores. Physical prediction Nov 2026 59.5 Not in cards
Unsatz Agents design experiments and propose equations for an unknown simulated system. Their final program must predict held-out interventions and conditions beyond the original experiments. Research science Oct 2026 60% Not in cards
AutoMedBench Agents set up and run medical imaging research workflows from a task brief through final submission. Scores combine task performance with rubric judgments of planning, setup, and validation, plus automatic checks of inference and submission. Machine learning engineering Jun 2026 66.5 Not in cards
Animation2Code Models recreate a web animation from a video using HTML, CSS, and JavaScript. The evaluator renders their code and compares appearance and motion separately. Web and app front ends May 2026 0.84 Not in cards
DexVerse Robot policies perform tabletop manipulation tasks in Isaac Lab, including tool use, bimanual coordination, and sequences with several goals. Task-specific success conditions check completion under different visual settings and robot embodiments. Simulated robots Jul 2026 34% Not in cards
Known-Unknown Questions Models distinguish questions with established answers from questions with no definitive answer. The authors score this distinction using F1 and equal error rate. Short factual questions May 2023 0.56 Not in cards
SciBench Models solve open-ended college-level problems in mathematics, chemistry and physics. Most answers are numbers, which a script checks against the key. Science questions Jul 2023 8.45% Not in cards
LLM-Coordination Models play four cooperative games and answer 198 multiple-choice questions about the same games. The authors report game outcomes and question accuracy separately. Multi-agent games Oct 2023 46.6 Not in cards
PitchBench Models listen to recordings and identify notes, intervals, chords, or melodies. The authors vary the instrument, timing, and background noise. Audio understanding and voice May 2026 19.8% Not in cards
ChatGPT Chemistry Assistant MOF synthesis extraction Models extract 11 synthesis details from papers about metal-organic frameworks. Researchers check the reported materials, amounts, temperatures, and reaction times against their annotations. Lab and data work May 2023 Not in cards
GD-VCR Models answer questions about what people are doing and why. The test pairs 328 images, mainly from film and television, with 886 questions across four regions. General visual questions Sep 2021 Not in cards
GeoMLAMA Models complete sentences about daily life in five countries. The test checks whether models rank locally correct answers above other choices in five languages. Knowledge in many languages May 2022 Not in cards
GS-QA2 Models answer questions about terrain and map features using elevation grids and vector data. The grader checks answers against PostGIS queries with tolerances for each measurement. Apps and MCP servers May 2026 Not in cards
LLM-NERRE materials extraction Models read materials papers and link dopants to host materials, guest species to metal-organic frameworks, and crystal structures to chemical formulas. Lab and data work Dec 2022 Not in cards
ULSA synthesis action retrieval Models identify and classify actions in sentences about inorganic materials synthesis. The corpus contains 3,040 labeled sentences from 535 paragraphs. Lab and data work Oct 2021 Not in cards