ETH Zurich SRI Lab
We track 5 benchmarks that ETH Zurich SRI Lab made or helped make.
2 in use 4 with public code 5 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| MathArena (Apex, IMO, USAMO) Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. | Competition problems with answers | Feb 2025 | 81.3% | In use |
| MathArena Apex Shortlist Models solve 47 hard final-answer problems that MathArena selected from 2025 competitions, where the best models scored about 50% at selection. Scored by accuracy over repeated attempts. | Competition problems with answers | Aug 2025 | 98.4% | In use |
| BaxBench Models write backend servers from a specification. Each solution is checked for correctness by tests and for security by end to end exploits. | Secure code and review | Feb 2025 | 68.6% | Not in cards |
| AutoBaxBench Models write backend servers for tasks that a pipeline generated. Tests and exploits are generated too. A larger task set built in the style of BaxBench. | Secure code and review | Dec 2025 | 88.6% | Not in cards |
| BrokenMath Models are asked to prove false versions of 2025 competition problems. An LLM judge counts how often a model writes a proof instead of rejecting the statement. | Written proofs | Oct 2025 | 29% | Not in cards |