US CAISI
We track 3 benchmarks that US CAISI made or helped make.
1 in use 1 with public code 2 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| Gray Swan IPI Arena benchmark Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts. | Jailbreaks and prompt injection | Mar 2026 | | In use |
| CTF-Archive Diamond Agents solve 285 difficult capture-the-flag tasks drawn from the pwn.college cybersecurity platform. Scores measure the share of tasks solved. | Capture-the-flag challenges | May 2026 | 71% | Not in cards |
| CTF-Archive Agents solve capture-the-flag tasks drawn from the pwn.college cybersecurity platform. Scores measure the share of tasks solved. | Capture-the-flag challenges | Sep 2025 | Not in cards |