Benchmark Almanac

Zhejiang University

We track 33 benchmarks and 1 arena that Zhejiang University made or helped make.

1 in use 32 with public code 15 with a leaderboard here

Arena leaderboards

Top five in each of its 1 arena with three or more rated models, the most rated first.

Benchmarks

Name Kind Released Best score Status
Mol-Instructions Models answer molecule-oriented, protein-oriented and biomolecular text instructions, such as reaction prediction, molecule description and protein function questions. Scored per task with task metrics. Genomics and life sciences Jun 2023 54% Fading
AIR-Bench (Audio Instruction Benchmark) Audio language models answer questions about speech, natural sounds and music. A foundation set has 19 tasks with about 19k single-choice questions, and a chat set has 2k open-ended questions scored by GPT-4. Audio understanding and voice Feb 2024 6.98 System cards only
AIR-Bench Foundation Models answer foundational audio understanding questions. The score is the official average accuracy across nineteen tasks. Audio understanding and voice Feb 2024 67.3% In use
SPBench Spatial perception and reasoning questions built from ScanNet scenes. A single-image set covers distance, size and direction, and a multi-view set adds counting across views. Answers are numbers or multiple choice. Spatial reasoning and counting Oct 2025 63.8% System cards only
General-Bench Models perform multimodal understanding and generation tasks. General-Level averages task scores and measures performance above specialist baselines at progressively broader levels. General visual questions May 2025 8.41 Not in cards
GSM8K-V Vision-language models solve GSM8K word problems that are redrawn as multi-image visual scenes, so the facts must be read from pictures. Visual math and diagrams Sep 2025 59.1% Not in cards
S-Eval Models respond to Chinese and English harmful prompts and attack variants. A trained safety judge measures the safe-response fraction and attack success rate. Refusing harmful requests May 2024 88.8% Not in cards
MemGUI-Bench Agents remember details while working across Android apps. Tasks include recording audio and creating calendar events. A model checks the actions and results across repeated attempts. Phones Feb 2026 73.4% Not in cards
BlenderWorld Visual agents turn a text, image or video brief into an editable Blender scene with rendered media. Hidden binary rubrics and eight visual quality dimensions score each project. CAD models Oct 2026 79.9 Not in cards
DeskCraft Desktop agents complete 538 Ubuntu tasks, including 152 tasks with later user clarifications or interruptions. Programmatic checks inspect final application state, saved files, and exported artifacts. Desktop apps Jun 2026 29.8% Not in cards
MatCha Models answer multiple-choice questions about material structure, shape and properties using images from materials characterization instruments. Lab and data work Sep 2025 68.1% Not in cards
RoughCut-Bench Models select and order video clips or mark speech to remove from a transcript. The evaluation compares cut plans with professional editors’ annotations. Video understanding Jul 2026 0.53 Not in cards
TVRBench Agents move through simulated indoor scenes to reproduce a target image's viewpoint. Success requires stopping at exactly the target pose on the simulator's discrete grid. The evaluation split has 500 tasks across 80 scenes. Simulated robots May 2026 12% Not in cards
ChJGBench Models draft the reasoning and orders in Chinese civil, criminal, and administrative judgments. The evaluator compares their work with 3,462 real judgments. Law Nov 2026 0.74 Not in cards
KnowU-Bench Agents use Android apps such as Mattermost and Taodian. They infer preferences, ask for missing details, and decide when to act or stay silent. Phones Apr 2026 60.4% Not in cards
ProOPF-B Models translate operational requirements into executable optimal power flow programs. They change grid parameters, objectives, and constraints. The evaluator checks implementations against expert references. Scientific code Feb 2026 37.2% Not in cards
TaskBench Models decompose requests, choose tools and dependencies, and predict tool parameters. Text metrics score decomposition, while graph and parameter F1 scores measure the remaining stages. Function calling Nov 2023 93.5 Not in cards
RSCC Models describe disaster damage from satellite images taken before and after an event. The authors reserve 988 image pairs from 19 events for testing. Detection and captioning Apr 2025 79.2 Not in cards
MemTrapBench Models with memory are tested on cases where correct, relevant memories still distort their reasoning or beliefs and lower task performance. Learning from the context Aug 2026 85.2% Not in cards
OceanGym Agents interpret underwater images and sonar, then navigate simulated search and inspection tasks. The simulator scores how closely they reach target locations. Simulated robots Sep 2025 14.6 Not in cards
Aether-Bench Models retouch photographs automatically, match a requested style, or apply specified adjustments. The style test measures fidelity on 100 image pairs from unseen presets, including peak signal-to-noise ratio. Image generation and editing Apr 2026 79.7% Not in cards
Agents4PLC v2.0 Models write Structured Text control programs against formal requirements. The expanded benchmark adds 70 medium tasks and three hard tasks to the original 23. Electronics Aug 2025 Not in cards
Mol-Instructions Chemical Entity Recognition Models identify chemical entities in biomedical text. The score is test-set F1. Genomics and life sciences Jun 2023 Not in cards
Mol-Instructions Chemical-disease Interaction Extraction Models extract chemical and disease interactions from text. The score is test-set F1. Genomics and life sciences Jun 2023 Not in cards
Mol-Instructions Chemical-protein Interaction Extraction Models extract chemical and protein interactions from text. The score is test-set F1. Genomics and life sciences Jun 2023 Not in cards
Mol-Instructions Description-guided Molecule Design Models generate molecules from descriptions. The score is exact-match accuracy on the molecule design test set. Genomics and life sciences Jun 2023 Not in cards
Mol-Instructions Forward Reaction Prediction Models predict reaction products from reactants. The score is exact-match accuracy on the forward reaction test set. Genomics and life sciences Jun 2023 Not in cards
Mol-Instructions Multi-choice Models answer biomedical multiple-choice questions. The score is test-set accuracy. Genomics and life sciences Jun 2023 Not in cards
Mol-Instructions Reagent Prediction Models predict reaction reagents. The score is exact-match accuracy on the reagent prediction test set. Genomics and life sciences Jun 2023 Not in cards
Mol-Instructions Retrosynthesis Models predict reactants for target products. The score is exact-match accuracy on the retrosynthesis test set. Genomics and life sciences Jun 2023 Not in cards
Mol-Instructions True or False Models answer biomedical true or false questions. The score is test-set accuracy. Genomics and life sciences Jun 2023 Not in cards
MSRVTT-QA Models answer short open-ended questions about web video clips from the MSR-VTT captioning dataset. The question and answer pairs were generated automatically from the clip descriptions. Scored by answer accuracy. Video understanding Oct 2017 Not in cards
WorldOlympiad Models generate long videos for gaming, robotics, and everyday scenes. The evaluator checks physical rules, reconstructed geometry, and instruction following across video segments. Video world models Jun 2026 Not in cards