Benchmark Almanac

York University

We track 3 benchmarks that York University made or helped make.

1 in use 3 with public code 2 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
ChartQA Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Documents, charts and text in images Mar 2022 90.9% In use
Ego-MC-Bench Models watch a cook follow spoken instructions and tell them when to correct a mistake. The evaluator checks intervention timing and recipe step completion. Video understanding Jun 2026 0.23 Not in cards
PoemSum Models summarize the meaning of an English poem. The authors compare each summary with a reference interpretation using ROUGE and BERTScore. Reading comprehension Dec 2023 Not in cards