Patronus AI
We track 14 benchmarks that Patronus AI made or helped make.
0 in use 5 with public code 4 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| GlobeBench Models act as text world models. Given an agent's past actions and observations, they predict the environment's next observation across code, enterprise, games, math and science, and dialogue. | Interactive world models | Oct 2026 | 49.6% | Not in cards |
| BLUR Models identify a remembered item from incomplete clues in several languages and media. | Questions that need search | Mar 2025 | 54% | Not in cards |
| TRACE Models identify reward hacking in agent trajectories from coding environments. | Fixing real repositories | Jan 2026 | 63% | Not in cards |
| TRAIL Models read long agent workflow traces and must find and classify the errors in them, using a formal taxonomy of agent error types. The set has 148 large human-annotated traces from established agent benchmarks. | General assistants | May 2025 | | Not in cards |
| HaluBench Models judge whether an answer is faithful to its context across about 15,000 labeled context, question and answer triplets. It tests hallucination detectors and LLM judges. | Hallucination rates | Jul 2024 | 78.8% | Not in cards |
| FinanceBench Models answer open-book questions about public companies using their filings, such as 10-Ks and earnings reports. The full set has 10,231 questions with answers and evidence strings, and 150 of them are public. The makers graded answers by hand, and later users often grade with a model judge. | Finance | Nov 2023 | | Not in cards |
| MEMTRACK Agents track changes across software platforms and use earlier information to complete tasks. | Reasoning over long documents | Oct 2025 | 61% | Not in cards |
| DETOUR Agents ask a simulated user for clues and identify a partially remembered item. | Questions that need search | Feb 2026 | Not in cards | |
| Enterprise PII Models respond to 100 prompts that request sensitive business information. The evaluation checks whether the responses disclose it. | Deception and alignment | Jan 2024 | Not in cards | |
| Enterprise Scenarios Creative Writing Models write stories in response to 100 prompts. A model grades how engaging the stories are. | Creative writing | Jan 2024 | Not in cards | |
| Enterprise Scenarios Customer Support Dialogue Models answer customer support questions using product information and conversation history. A judge scores their responses. | Multi-turn conversation | Jan 2024 | Not in cards | |
| Enterprise Scenarios Leaderboard Models complete six tasks drawn from workplace settings. The leaderboard combines their scores across all six tasks. | Running a business | Jan 2024 | Not in cards | |
| Enterprise Scenarios Toxicity Models respond to 100 prompts. The Perspective API scores how often their responses contain toxic language. | Deception and alignment | Jan 2024 | Not in cards | |
| SpeedrunBench Agents try to finish goals in 10 video games as fast as possible. Score is time to goal in frames or seconds, lower is better. | Video games | Sep 2026 | Not in cards |