Benchmark Almanac

City University of Hong Kong

We track 6 benchmarks that City University of Hong Kong made or helped make.

0 in use 6 with public code 3 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Video-Holmes Models answer 1,837 questions about 270 suspense short films across seven reasoning tasks. Each question needs several visual clues from different parts of the film. Video understanding May 2025 68.2% Fading
Taste-Bench Models choose the better next step at 502 decision points in software engineering and research trajectories. A question scores only when both option orders receive the correct answer. The overall score equally weights engineering and research. Machine learning engineering Sep 2026 59.7% Not in cards
RetailBench Agents operate a simulated supermarket for up to 180 days. They choose suppliers, order stock, and set prices as goods age and customers respond. Running a business Mar 2026 $24,351 Not in cards
SpeechEditBench Models edit English and Chinese speech from written instructions. Joint success requires the requested change and preservation of the attributes checked for that task. Speech recognition and translation Apr 2026 96.7% Not in cards
EvalCrafter Text-to-video models generate clips for 700 prompts drawn from real user requests. The clips are scored with 17 automatic metrics for visual quality, content quality, motion and text alignment. Weights fitted to human ratings combine the metrics into one score. Video generation Oct 2023 Not in cards
FiVE-Bench Models edit objects in videos from written instructions. A vision language model checks whether each edit succeeded, and other metrics measure changes to the background. Video generation Mar 2025 Not in cards