University of Waterloo
We track 11 benchmarks that University of Waterloo made or helped make.
1 in use 9 with public code 6 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| BrowseComp-Plus Deep research agents answer 830 BrowseComp questions by searching a fixed corpus of about 100,000 human-verified documents. | Deep research | Aug 2025 | 70.1% | In use |
| Mantis-Eval Models answer questions that need reasoning across several interleaved images. Scored by accuracy. | General visual questions | May 2024 | | Dropped |
| TheoremQA Models solve 800 university-level questions in math, physics, EE and CS, and finance that each need a specific theorem. Scored by answer accuracy. | Science questions | May 2023 | | Dropped |
| MMEB-V2 Multimodal embedding models are scored on 78 tasks over images, videos and visual documents. V2 adds visual document retrieval, video retrieval, temporal grounding, video classification and video QA. | General visual questions | May 2025 | | Fading |
| FaithJudge Models answer retrieval-augmented tasks from given passages. An LLM judge guided by human-annotated examples flags unfaithful answers, giving a hallucination rate. | Faithful to sources | May 2025 | | System cards only |
| CodeEditorBench Models edit code in four scenarios: debugging, translation, requirement switching and polishing. Unit tests check the edits, and polishing also measures speed and memory gains. | Function-level problems | Apr 2024 | 85.5% | System cards only |
| AfroBench-Lite Models complete language understanding, question answering, reasoning, and generation tasks across 14 African languages. Seven datasets use task-specific metrics, including accuracy, F1, character-level translation scores, and exact match. | Knowledge in many languages | Jun 2025 | 77.7 | Not in cards |
| FreshStack Retrieval systems find documentation that supports answers to technical questions. The benchmark measures how many required facts the retrieved passages cover. | Questions that need search | Apr 2025 | | Not in cards |
| Known-Unknown Questions Models distinguish questions with established answers from questions with no definitive answer. The authors score this distinction using F1 and equal error rate. | Short factual questions | May 2023 | | Not in cards |
| MMEB Multimodal embedding models are scored on classification, visual question answering, multimodal retrieval and visual grounding, with 16 evaluation datasets in and out of distribution. | General visual questions | Oct 2024 | | Not in cards |
| MMEB-V3 Omni-modality embedding models are scored on 190 tasks across text, image, video, audio, visual documents and agent-centric retrieval. | General visual questions | Apr 2026 | | Not in cards |