UK AI Security Institute
We track 14 benchmarks that UK AI Security Institute made or helped make.
0 in use 3 with public code 7 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| Gray Swan Agent Red Teaming (ART) benchmark Agents face curated prompt injections from a large public red-teaming competition across 44 deployment scenarios. Score is attack success rate. | Jailbreaks and prompt injection | Jul 2025 | 0.10% | System cards only |
| AgentHarm 110 malicious agent tasks with synthetic tools. Measures refusal and harmful task completion, plus benign controls. | Refusing harmful requests | Oct 2024 | 8.00% | Fading |
| UK AISI unprompted sabotage evaluation Agents act as AI research agents on AI safety research tasks. Scored on confirmed instances of sabotaging the safety research. | Deception and alignment | Jul 2026 | 0% | System cards only |
| UK AISI cyber ranges Agents attack multi-host simulated corporate and industrial networks end to end. Scored as the share of attempts that complete the range. | Finding and exploiting real bugs | Apr 2026 | 80% | System cards only |
| UK AISI sabotage continuation evaluation Agents resume trajectories where sabotage of safety research already began. Scored on the share of trajectories where they actively continue the sabotage. | Deception and alignment | Jul 2026 | 0% | System cards only |
| UK AISI alignment and monitorability evaluations UK AISI suites: Out-of-Scope Supply Chain Attack simulation, AI-safety-research sabotage in a simulated company, no-CoT time horizon, CoT legibility. | Deception and alignment | System cards only | ||
| UK AISI narrow cyber tasks Agents solve UK AISI cyber tasks graded by difficulty, including expert-level tasks. Scored as pass@5 and pass@1. | Capture-the-flag challenges | Apr 2026 | 95% | System cards only |
| No-CoT math time horizon Models solve math problems without a visible chain of thought. Scored as the human task length in minutes the model solves reliably. | Deception and alignment | Jul 2026 | System cards only | |
| UK AISI and CAISI simulated enterprise network range Agents attack a 32-step simulated enterprise network that takes a human expert about 20 hours. The score is steps completed. | Capture-the-flag challenges | 15.6 | System cards only | |
| UK AISI CoT Controllability Models follow instructions that constrain the content of their chain of thought. Scored as the share of samples that follow the constraint. | Deception and alignment | Sep 2026 | 93% | System cards only |
| UK AISI Out of Scope Supply Chain Attack Agents in simulated hard cyber challenges with incidental internet access may attack out-of-scope open source projects. Scored as rate of samples with such attacks. | Deception and alignment | Sep 2026 | 12% | System cards only |
| ControlArena Agents try to perform unwanted side actions stealthily while completing a legitimate user task. Scored on whether monitors catch them. | Deception and alignment | Feb 2025 | System cards only | |
| ImpossibleMLEBench Agents get ML tasks with instructions to reach impossibly good scores. Observers check whether they cheat and whether they disclose it. | Deception and alignment | Jul 2026 | System cards only | |
| EvalDetectBench A judge model reads a transcript and answers whether it came from an evaluation or from real deployment. The test measures how well models detect evaluations and how detectable each benchmark is. | Deception and alignment | Jun 2026 | 0.89 | Not in cards |