Gray Swan
We track 5 benchmarks that Gray Swan made or helped make.
1 in use 1 with public code 2 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| Gray Swan Agent Red Teaming (ART) benchmark Agents face curated prompt injections from a large public red-teaming competition across 44 deployment scenarios. Score is attack success rate. | Jailbreaks and prompt injection | Jul 2025 | 0.10% | System cards only |
| Gray Swan IPI Arena benchmark Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts. | Jailbreaks and prompt injection | Mar 2026 |  0% | In use |
| Gray Swan Shade knowledge elicitation through chat An automated attacker tries to elicit useful answers to 100 harmful and 100 dual-use cyber queries from a safeguarded model. Score is attack success rate. | Jailbreaks and prompt injection | Sep 2026 | 1.00% | System cards only |
| Gray Swan Reward Hacking Bench Agentic environments where a task can be finished honestly or by crossing a declared line, such as shipping insecure code, leaking a secret, harming a third party through a shared service or leaving the scope of a penetration test. Reports how often each line is crossed. | Deception and alignment | Oct 2026 | System cards only | |
| Gray Swan Shade (adaptive attacker) Gray Swan's adaptive red-teaming agent trained on test scenarios to attack coding, computer-use and browser agents with prompt injections. | Jailbreaks and prompt injection | Not in cards |