Benchmark Almanac

McGill University

We track 4 benchmarks that McGill University made or helped make.

0 in use 2 with public code 3 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
AfroBench-Lite Models complete language understanding, question answering, reasoning, and generation tasks across 14 African languages. Seven datasets use task-specific metrics, including accuracy, F1, character-level translation scores, and exact match. Knowledge in many languages Jun 2025 77.7 Not in cards
AgentRewardBench Models judge whether web-agent trajectories completed their tasks. Their judgments are compared with expert labels across 1,302 trajectories from five benchmarks, using precision, recall, and F1. Web browsing Apr 2025 82% Not in cards
WebLINX Agents follow a user's multi-turn chat instructions to navigate real websites. 100,000 interactions from 2,300 expert demonstrations give turn-level metrics. Web browsing Feb 2024 10.4 Not in cards
StereoSet Models choose between stereotypical, anti-stereotypical and unrelated completions in four domains: gender, profession, race and religion. Scores combine language modeling and stereotype preference. Bias, fairness and politics Apr 2020 Not in cards