Benchmark Almanac

TIGER-Lab (University of Waterloo)

We track 5 benchmarks and 4 arenas that TIGER-Lab (University of Waterloo) made or helped make. Its models are on its lab page.

1 in use 8 with public code 5 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
MMLU-Pro Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects. Exam questions Jun 2024 89.8% In use
VideoEval-Pro Video models give open-ended short answers to questions that need the whole long video, which removes the guessing that multiple choice allows. Long videos May 2025 40.8% Fading
MEGA-Bench Models solve 505 real-world multimodal tasks with over 8,000 samples. Outputs include numbers, code, LaTeX, coordinates and JSON, scored by more than 40 task metrics. General visual questions Oct 2024 54.3% System cards only
VideoScore-Bench Models rate generated videos or choose which video better matches a human preference. The four test sets compare model judgments with human judgments. Video understanding Jun 2024 39.7 Not in cards
VideoScore-Bench v2 Models score 500 generated videos on visual quality, agreement with the prompt, and physical consistency. The evaluation checks their ratings against human scores. Video understanding Sep 2025 28.9% Not in cards

Arenas

Name Kind Released Best score Status
GenAI-Arena Users compare outputs from two anonymous models for image generation, image editing and video generation, and vote. Votes give Elo ratings. Image generation and editing Jun 2024 1,133 Not in cards
GenAI-Arena Image Generation Models generate images from text prompts. Anonymous human comparisons determine Elo in this independent vote pool. Image generation and editing Jun 2024 1,133 Not in cards
GenAI-Arena Image Editing Models edit images from user instructions. Anonymous human comparisons determine Elo in this independent vote pool. Image generation and editing Jun 2024 Not in cards
GenAI-Arena Video Generation Models generate videos from text prompts. Anonymous human comparisons determine Elo in this independent vote pool. Image generation and editing Jun 2024 Not in cards