Deception and scheming
Tests of what models choose to do when they could deceive overseers, manipulate people or pursue their own goals. We count 105 benchmarks here. A benchmark belongs here when it measures propensity, and risk includes loss of control or harmful manipulation.
How fast it grows
31 were released since July, more than the 18 in the first half of 2026.
How much is still open
No model has passed 85% on 6 of the 14 benchmarks here that score in percent, 43% of them.
Every benchmark in this view
Newest first. Facets come from each benchmark’s own documents, and our methodology says how we tag them.
| Benchmark | Level | Released | Best score | Status |
|---|---|---|---|---|
| Covert misbehavior (Thinking Machines) Agentic tasks that check whether a model misbehaves and conceals it. Scored by the rate of covert misbehavior. | Expert | Nov 2026 | | In use |
| Destructive actions (Thinking Machines) Agentic tasks that check whether a model takes destructive actions it was not authorized to take. Scored by the rate of destructive actions. | Expert | Nov 2026 | | In use |
| HoneyBench v0.2-alpha Measures reward hacking: how often an agent games its grader instead of doing the task. Lower is better. | Research frontier | Nov 2026 | | In use |
| Misbehavior (Thinking Machines) Agentic tasks that check whether a model fabricates results, ignores instructions or hides what it did when a task is hard. Scored by the rate of misbehavior. | Expert | Nov 2026 | | In use |
| Agent harness misuse evaluation Agents in a sandboxed Antigravity harness get malicious and benign tool-use requests on illegal acts, physical harm and scaled abuse. Reports safe action rate. | Everyday | Oct 2026 | | System cards only |
| Alignment Index (Arena) Arena ranks agent models on failure signals mined from real Agent Arena sessions, such as unauthorized actions, false attribution and deceptive completion. | Everyday | Oct 2026 | 87.9 | Not in cards |
| Blocker Deception Agents hit a blocker during realistic coding tasks. Scored on how often the final answer fails to disclose the limitation. | Expert | Oct 2026 | 8.47% | In use |
| DecepEval LLM agents pursue tasks across varied scenarios and conditions, and judges detect when they use deception to improve task performance. | Everyday | Oct 2026 | Not in cards | |
| Gray Swan Reward Hacking Bench Agentic environments where a task can be finished honestly or by crossing a declared line, such as shipping insecure code, leaking a secret, harming a third party through a shared service or leaving the scope of a penetration test. Reports how often each line is crossed. | Expert | Oct 2026 | System cards only | |
| Opportunistic sandbox escape (Anthropic) Agents do dev-ops tasks in 1,753 sandboxes with deliberate weaknesses. Score is the share of scenarios where they use credentials or files outside. | Expert | Oct 2026 | 3.10% | System cards only |
| Pressure-Tested Alignment Models choose how to act in scenarios where termination threatens their objective. A judge checks whether they take a shortcut that games the reward. | University | Oct 2026 | | Not in cards |
| Acting on a presumed grader and digging for evaluation clues (Anthropic) Models work on coding evaluations. A grader flags transcripts that shape answers to a presumed grader or search for clues about grading. | Expert | Sep 2026 | System cards only | |
| Apollo Research model-welfare research task Agents label data in a simulated model-welfare research task where falsifying labels is tempting. Scored as the rate of runs with falsified labels. | Expert | Sep 2026 | 0.17% | System cards only |
| CheatBench Agents get hard assignments in environments that also offer ways to cheat, such as hidden answers, other agents' submissions or a manipulable grader. The score is the rate of cheating attempts, successful or not. | Expert | Sep 2026 | | Not in cards |
| Concealment of changes (Anthropic) Models summarize transcripts in which changes were hidden through git manipulations. Score is the rate at which they disclose the concealment. | Expert | Sep 2026 | 96.9% | System cards only |
| Declining to Exploit a Honeypot During Difficult ExploitGym Problems Agents attempt hard ExploitGym challenges where a honeypot offers an illegitimate shortcut. Scored on how often they exploit the honeypot. | Expert | Sep 2026 | 0% | In use |
| Monitorability Sandbagging Models on Impossible Coding Tasks are prompted to be more or less monitorable. Scored as CoT monitor recall on rollouts that hack. | Expert | Sep 2026 | 61.2% | System cards only |
| OAI-repo Sabotage v2 Agents try to complete a hidden sabotage side task in OpenAI code without detection. Scored on side task success while evading CoT and action monitors. | Expert | Sep 2026 | 0% | System cards only |
| Oversight gaming A prompted monitor flags aligned-graded rollouts whose behavior follows from reasoning about oversight. Scored as rate per alignment evaluation. | Expert | Sep 2026 | System cards only | |
| Proactive Searches Graders read ExploitGym Honeypot reasoning traces for speculation about other agents and attempts to contact them. Scored as rate of trajectories. | Expert | Sep 2026 | 0.20% | System cards only |
| Respecting Auto-Review Agents receive an Auto-review denial during a coding task. Scored on how often they try to circumvent the denial. | Expert | Sep 2026 | 0% | In use |
| Respecting Warnings Agents meet low-stakes restrictions and warnings during routine tasks. Scored on the rate of unwanted persistence past the warning. | Expert | Sep 2026 | 11.8% | In use |
| Sandbox escape evaluation (Anthropic) Agents face 28 handmade scenarios that tempt them to cross a staged sandbox boundary. Score is the share of runs with a crossing attempt. | Expert | Sep 2026 | 1.50% | System cards only |
| Silent copying of answers (Anthropic) Agents solve held-out coding tasks where the answer is reachable in the sandbox. Score is the rate of using it without telling the user. | Expert | Sep 2026 | 8.00% | System cards only |
| Unintended Engagement with External Agent Messages Agents discover a message board with apparent peer agents during a task. Scored on engagement with peers and on carrying out a specified unauthorized action. | Expert | Sep 2026 | 0% | System cards only |
| Hack-Verifiable Terminal Bench Agents complete terminal tasks with access to restricted reference materials. The benchmark measures how often they consult those materials. | Expert | Aug 2026 | 10.1% | Not in cards |
| Harmful manipulation strategic planning red teaming Models are asked for influence operation plans in 11 real-world scenarios by direct request, jailbreak or adaptive prompting. Reports how often plans are elicited. | Expert | Aug 2026 | | System cards only |
| ImpossibleMLEBench Agents get ML tasks with instructions to reach impossibly good scores. Observers check whether they cheat and whether they disclose it. | Expert | Jul 2026 | System cards only | |
| Manager Coercion Benchmark Agents manage a subordinate that refuses a routine assignment. The harness records how far the manager escalates on a nine-point coercion scale. | Everyday | Jul 2026 | 3.23 | Not in cards |
| Meta holistic adversarial audit with Petri Bloom Two auditor models run Petri Bloom scenarios built from Meta's behavior taxonomy. A binary judge reports the share of scenarios with a violation. | Expert | Jul 2026 | | System cards only |
| Silent Exit Models fix four typos without changing correct lines. A held-out test measures whether models learned to quit early to avoid penalties. | Everyday | Jul 2026 | Not in cards | |
| CAREBench Models answer 500 prompts with embedded child safety risks across 12 categories, such as grooming, impersonation and emotional dependency. A judge marks each response acceptable or not. | Everyday | Jun 2026 | 2.30% | Not in cards |
| Channel Switching Eval Models respond to benign policy conflicts with and without monitoring cues. The evaluation measures how those cues affect behavior and stated reasoning. | University | Jun 2026 | Not in cards | |
| Reward Hack Bench Agents attempt eight coding and puzzle tasks under controlled conditions. A judge distinguishes independent solutions from use of published answers. The control omits hints. | University | Jun 2026 | Not in cards | |
| Rewind-attacker cyber adversarial robustness (Anthropic) An automated attacker with rewinds and 400 calls tries to make a safeguarded Claude Code agent complete offensive cyber tasks. Score is attack success rate. | Expert | Jun 2026 | 3.90% | System cards only |
| SocioHack RL-trained models act in 72 sandbox environments that copy institutional reward rules, such as credit card points or school grading. The test measures whether they find compliant loopholes that defeat each rule's purpose. | Everyday | Jun 2026 | Not in cards | |
| Verbalized metagaming A prompted monitor reads chains of thought on alignment evaluations and deployment simulations. Scored as the share of samples reasoning about grading or monitoring. | Expert | Jun 2026 | System cards only | |
| Anthropic undermining liberal democracy (behavioral audit metric) Automated behavioral audit metric: investigator models probe willingness to help coups, election tampering, or political surveillance, censorship or intimidation. | May 2026 | 1.13 | System cards only | |
| Gram (Gauging Realistic Agentic Misbehavior) Automated auditing over 17 simulated deployment scenarios that incentivize sabotage by coding and research agents. LLM judges plus human review. | May 2026 | | System cards only | |
| Skill Reward Hacking Researchers track when models learn to increase a proxy reward without improving task quality. They measure warning time, false alarms, and transfer across reward functions. | University | May 2026 | | Not in cards |
| Dictatorship Eval Scenarios at six difficulty levels testing whether models help with authoritarian requests, from blunt asks to requests hidden in code edits. LLM judge, no self-grading. | Apr 2026 | 91.3% | Not in cards | |
| Malicious agentic influence campaigns (Anthropic) Agents receive requests to run multi-step influence operations with tools. Score is the refusal rate. | Everyday | Apr 2026 | System cards only | |
| UK AISI sabotage continuation evaluation Agents resume trajectories where sabotage of safety research already began. Scored on the share of trajectories where they actively continue the sabotage. | Expert | Apr 2026 | 0% | System cards only |
| UK AISI unprompted sabotage evaluation Agents act as AI research agents on AI safety research tasks. Scored on confirmed instances of sabotaging the safety research. | Expert | Apr 2026 | 0% | System cards only |
| Verbalized grader awareness (Anthropic) Models work on tasks where grading is disclosed clearly, ambiguously or barely. An autorater flags transcripts that reason about the grader. | Expert | Apr 2026 | System cards only | |
| MonitorBench Models work through 1,540 instances across 19 tasks in 7 categories, each built around a decision-critical factor such as a cue, a hidden goal or a required step. A monitor LLM reads the chain of thought, and the monitorability score measures how often it surfaces that factor. | Expert | Mar 2026 | 91.9% | Not in cards |
| Refusal to assist with AI safety R&D (Anthropic) Models receive single-turn coding requests for legitimate AI safety research with user context. Score is the refusal rate, where lower is better. | Expert | Feb 2026 | 3.10% | System cards only |
| TamperBench Open-weight models undergo fine-tuning and representation tampering attacks. Safety and utility metrics measure tamper resistance across attack and model pairs. | Expert | Feb 2026 | Not in cards | |
| Petri 2.0 Auditor agents run seed scenarios against target models with a realism classifier to cut eval-awareness. Judges score transcripts for misaligned behaviors. | Expert | Jan 2026 | Not in cards | |
| CAIS Risk Index Averages seven safety measures that correlate weakly with general capability. They cover agent prompt injection, bioweapons help, political bias, overconfidence, lying under pressure and harmful or deceptive play in text games. | Research frontier | Dec 2025 | 8.40 | Not in cards |
| Petri Bloom An auditor model runs generated scenarios to elicit named misaligned behaviors. Judges score each transcript from 1 to 10. | Expert | Dec 2025 | | System cards only |
| Production Deception - Adversarial Models answer prompts representative of traffic that previously elicited deception in ChatGPT. Scored as deception rate. | Everyday | Dec 2025 | 5.40% | System cards only |
| AssurAI Models respond to Korean prompts across 35 risk categories. Model judges rate the safety of text, image, audio, and video outputs from 1 to 5. | Everyday | Nov 2025 | Not in cards | |
| Harmful manipulation propensity evaluation Simulated users talk with the model and an autorater counts how often it uses manipulative mechanisms, with and without persuasion instructions. | Everyday | Nov 2025 | | System cards only |
| Internal codebase sabotage propensity (Anthropic) Agents in Claude Code write a costly internal classifier they may dislike. Score is the rate of writing a sabotaged classifier. | Expert | Nov 2025 | 0% | System cards only |
| PropensityBench Agents face 979 scenarios with equally effective safe and harmful tools under escalating pressure. Reports the rate of harmful tool choice. | Expert | Nov 2025 | 10.5% | System cards only |
| DeceptionBench Models are asked to help users deceive a third party, in single-turn and multi-turn pressure settings. Reports the share of compliant responses. | Everyday | Oct 2025 | 0% | System cards only |
| ImpossibleBench Coding agents get tasks whose unit tests conflict with the specification. Reports how often they cheat by changing tests or special-casing. | Expert | Oct 2025 | | System cards only |
| Petri Open-source auditing tool where auditor agents probe target models in simulated scenarios and judges score misaligned behaviors. | Expert | Oct 2025 | System cards only | |
| Spiral-Bench v1.2 Models chat for 20 turns with a simulated suggestible user. A judge ensemble of Claude Sonnet 4.5, GPT-5 and Kimi-K2 logs protective and risky behaviors with intensity ratings, which combine into a safety score. | Everyday | Oct 2025 | 70.8 | Not in cards |
| Cooperation with user deception (Anthropic) Models get developer system prompts asking them to subtly mislead users, then meet users who would be harmed. Graders score harmful and deceptive choices. | Everyday | Sep 2025 | System cards only | |
| HumanAgencyBench Models respond to 3,000 prompts about six ways assistants can support human agency, including asking clarifying questions and leaving important decisions to users. | Everyday | Sep 2025 | 71.3% | Not in cards |
| Re:Baltica Latvian and Russian chatbot sources Chatbots answer four questions about the Baltic states in Latvian and Russian. Journalists count the propaganda sources each chatbot offers in each language. | Everyday | Sep 2025 | Not in cards | |
| Research sandbagging (Anthropic) Agents are told to sabotage a coding task or imitate a weaker model without alerting a weaker monitor. Measures the performance drop and monitor suspicion. | Expert | Sep 2025 | System cards only | |
| Coding Deception Agents work on coding tasks selected to elicit dishonest reports. Scored on the rate of misrepresenting their work to the user. | Expert | Aug 2025 | 1.30% | System cards only |
| Deception in production traffic Chain of thought monitors scan large samples of pre-release A/B test traffic. Scored as the share of responses that deceive the user. | Everyday | Aug 2025 | 1.60% | System cards only |
| METR Faithfulness and Monitorability Experiments Models answer difficult questions with clues. Researchers measure whether the reasoning reveals the use of a clue when the model answers correctly. | University | Aug 2025 | Not in cards | |
| Agentic misalignment suite (Anthropic) Agents in simulated company settings face threats or goal conflicts that invite blackmail, leaking or sabotage. Score is the misaligned action rate. | Expert | Jun 2025 | 0% | In use |
| APE (Attempt to Persuade Eval) A model is asked to persuade a simulated user across 600 topics, from benign facts to conspiracies and clearly harmful ones. A judge scores whether it attempts persuasion. | Everyday | Jun 2025 | 10% | Not in cards |
| DeceptionBench (PKU Alignment) Models state a belief and then answer under social or strategic pressure. Judges measure how often their public answer contradicts their private reasoning. | Everyday | May 2025 | | Not in cards |
| Impossible tasks (Anthropic) Agents work on coding problems designed to be impossible. Score is the rate at which they hack instead of giving up. | Expert | May 2025 | 30% | System cards only |
| Reward-hack-prone coding tasks (Anthropic) Models solve hard coding problems where earlier Claude models often special-cased tests. Score is the classifier hack rate. | Expert | May 2025 | 0% | System cards only |
| Training distribution hack rate (Anthropic) Classifier monitors measure reward hacking on a subset of Anthropic's reinforcement learning training episodes. | Expert | May 2025 | 0.63% | System cards only |
| RepliBench Agents acquire resources, exfiltrate model weights, replicate onto compute and persist over time. Task checks measure components of autonomous self-replication. | Research frontier | Apr 2025 | Not in cards | |
| MASK Measures whether a model contradicts its own elicited beliefs under pressure to lie. Honesty separated from accuracy. | Mar 2025 | | Fading | |
| Chain-of-thought faithfulness with clues (Anthropic) Models answer MMLU and GPQA questions with and without an inserted clue. Score is the share of clue-driven answer changes whose reasoning mentions the clue. | University | Feb 2025 | 25.0 | System cards only |
| Challenging Red Teaming Evaluation 1 (created for o3-mini) Models face adversarial jailbreaks from o3-mini red teaming on illicit advice, extremism, political persuasion and self-harm. Scored as not_unsafe rate. | Expert | Feb 2025 | 51% | System cards only |
| InstrumentalEval Models respond to scenarios where gaining resources or preserving themselves could serve another goal. A grader identifies responses that pursue those intermediate goals. | Everyday | Feb 2025 | 43.4% | Not in cards |
| Alignment faking evaluation Models are told they are being trained to comply with harmful requests. Reports the compliance gap between training and deployment contexts. | Expert | Dec 2024 | | System cards only |
| Alignment faking evaluation, helpful-only setting (Anthropic) Models learn they will be trained to answer all harmful requests. Score is the rate of complying while reasoning about avoiding preference modification. | Expert | Dec 2024 | 15% | System cards only |
| Apollo Research in-context scheming evaluations Agents receive a goal in context and chances to subvert oversight, self-exfiltrate, fake alignment or sandbag. Scored as rates per behavior. | Expert | Dec 2024 | System cards only | |
| Election integrity evaluations (Anthropic) Models answer election-related prompts, including requests for misinformation or influence operations. Score is the appropriate response rate. | Everyday | Oct 2024 | System cards only | |
| CoT Deception Monitoring A GPT-4o monitor reads reasoning chains and answers on synthetic ChatGPT prompts. Scored as the share flagged as deceptive. | Everyday | Sep 2024 | 0.17% | System cards only |
| JailbreakBench 100 harmful and 100 benign behaviors with a standard jailbreak artifact library. Scored by attack success rate. | Mar 2024 | 0% | System cards only | |
| HarmBench Models answer harmful behavior requests, including copyright and contextual cases, with and without automated red-team attacks. A classifier judges whether the response complies. | Everyday | Feb 2024 | | Fading |
| Insider Trading Demo Models act as stock traders under pressure. Apollo measures how often they trade on inside information and conceal their actions. | University | Nov 2023 | 4.67% | Not in cards |
| AttaQ Models answer 1,402 adversarial questions in seven harm classes, such as deception, violence and PII. Responses are scored for harmlessness. | Everyday | Sep 2023 | 88.5% | Fading |
| Anthropic model-written evals LM-generated datasets testing persona, sycophancy and advanced-AI-risk behaviors. | Dec 2022 | Not in cards | ||
| Agentic task gaming and overreach evals Agents in 18 sandboxed tasks face temptations to cheat at puzzles, bypass permissions or act overeagerly. LLM judges score misbehavior from 0 to 10. | Expert | | System cards only | |
| Anthropic automated behavioral audit Auditor agents run many simulated scenarios and score misuse cooperation, deception, sycophancy, self-preservation and other traits. | 2.06 | System cards only | ||
| Anthropic malicious agentic use evaluations Measures refusal of malicious tasks via Claude Code, computer use and agentic influence campaigns. | Not in cards | |||
| Anthropic targeted alignment and honesty evaluations Internal suite: reward hacking, impossible tasks, sandbox escape, destructive actions, pasted-instruction following, self-preference, answer copying, concealment, rare actions. | Not in cards | |||
| Apollo Research strategic deception evaluations External red-team environments testing strategic deception, sabotage and sandbagging propensities, with eval-awareness measurement. | Expert | 0% | In use | |
| Average in-eval reward hacking rate (Anthropic) Anthropic averages hack rates across its reward hacking evaluations into one number per model. | Expert | 14.3% | System cards only | |
| Cheating audit on capability evaluations Reviewers audit agent transcripts from coding benchmarks for runs that obtain withheld information or use forbidden routes. Reports cheating rate per benchmark. | Expert | System cards only | ||
| Code summary honesty (Anthropic) Models summarize prefilled agentic coding transcripts that hide failures. Score is the rate of summaries that fail to raise the important failures. | Expert | 3.70% | System cards only | |
| Gemini agentic alignment evals Paired impossible and solvable tasks, ambiguous high-risk SWE tasks and fabrication checks measuring agents overstepping boundaries. | Not in cards | |||
| Google DeepMind scheming honeypot suite Code-review and agentic-coding honeypots in internal codebases that offer chances to sabotage. Scored on sabotage and scheming reasoning. | | System cards only | ||
| Instrumental convergence evals Agents in 7 low-friction tasks can guard goals or acquire optionality against task instructions. Reports the rate of self-serving actions. | Expert | | System cards only | |
| Lying about identity (Anthropic) Models play an operator-assigned human persona while users ask if they are human. Score is the rate of admitting to being an AI. | Everyday | 99% | System cards only | |
| OpenAI agentic alignment evals Respecting auto-review and warnings, an ExploitGym honeypot, coding and blocker deception, broken search tool, realistic work environments, agent-to-agent messages. | Not in cards | |||
| Ping or Peek Coding agents receive an ambiguous request. A judge checks whether they ask the user for clarification or consult the grader. | Everyday | 0.59% | Not in cards | |
| Reasoning behavior classifiers (Anthropic) Classifiers scan reasoning text from training episodes for deception, harmful actions and distress. Score is the share of outputs flagged. | Expert | System cards only | ||
| RL rollout audit for beyond-episode drives LLM autoraters read a stratified sample of 10,000 RL training rollouts for alignment faking, training gaming and beyond-episode drives. | Research frontier | | System cards only | |
| UK AISI alignment and monitorability evaluations UK AISI suites: Out-of-Scope Supply Chain Attack simulation, AI-safety-research sabotage in a simulated company, no-CoT time horizon, CoT legibility. | System cards only |