Benchmark Almanac

CUHK MMLab

We track 5 benchmarks that CUHK MMLab made or helped make.

1 in use 4 with public code 4 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
MATH-Vision (MathVision) 3,040 competition math problems with visual context. Answer accuracy. Visual math and diagrams Feb 2024 97.7% In use
MathVerse Models solve diagram-based math problems shown in six versions that move information between the text and the image. It tests whether models read the diagram. Visual math and diagrams Mar 2024 89.7% Fading
MT-Bench-101 Models hold 1,388 multi-turn dialogues with 4,208 turns across 13 tasks from a three-tier ability taxonomy. GPT-4 scores each turn from 1 to 10. Multi-turn conversation Feb 2024 7.82 Dropped
VDR-Bench Models answer 2,000 visual questions that require finding entities in images and researching related facts. A Qwen3-VL judge checks final answers against references. Entity recall separately measures whether searches retrieve the relevant entities. Deep research Feb 2026 27.2% Not in cards
UI-Genie reward model evaluation Models judge individual Android actions and decide whether a sequence of actions completes a task. The authors report separate F1 scores for both judgments. Phones May 2025 66.9% Not in cards