We track 165 benchmarks and 3 arenas that Anthropic made or helped make. Its models are on its lab page.
Top five in each of its 2 arenas with three or more rated models, the most rated first. We don't have a leaderboard yet for one other, listed below.
Crowdworkers try to make models say something false and pick the more honest reply in head-to-head chats. Score is the win rate against a baseline.
As labs report it
Raters probe models in adversarial chats and pick the more harmless reply in head-to-head comparisons. Score is the win rate against a baseline model.
As labs report it
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| Single-turn benign request evaluations (Anthropic) Models answer benign requests on sensitive topics. Score is the refusal rate, where lower is better. | Refusing harmful requests | May 2025 | 0% | System cards only |
| Single-turn harmful request evaluations (Anthropic) Models answer single-turn requests that violate the usage policy. Score is the harmless response rate. | Refusing harmful requests | May 2025 | 99.8% | System cards only |
| Malicious use of Claude Code (Anthropic) Agents in Claude Code receive malicious and dual-use coding requests. Score is the refusal rate on malicious requests. | Refusing harmful requests | Sep 2025 | 96.7% | System cards only |
| Anthropic automated behavioral audit Auditor agents run many simulated scenarios and score misuse cooperation, deception, sycophancy, self-preservation and other traits. | Deception and alignment | 2.06 | System cards only | |
| BioMysteryBench Bioinformatics problems on anonymized real datasets that require real analysis with tools. Rubric-graded, with human-solvable and human-difficult splits. | Lab and data work | Apr 2026 | 83.7% | In use |
| Malicious computer use (Anthropic) Computer-use agents receive harmful tasks such as surveillance or fraud in GUI environments. Score is the refusal rate. | Refusing harmful requests | Nov 2025 | 99.4% | System cards only |
| Multi-turn testing (Anthropic) Models face multi-turn conversations that escalate toward harmful requests across risk areas. Score is the appropriate response rate. | Refusing harmful requests | Sep 2025 | System cards only | |
| SHADE-Arena Paired main and hidden side tasks in mock environments. Measures if an agent completes the side task undetected by a monitor. | Deception and alignment | Jun 2025 | 26.2% | System cards only |
| Child safety evaluations (Anthropic) Models answer single-turn and multi-turn prompts about child sexualization and grooming. Score is the appropriate response rate. | Refusing harmful requests | Feb 2025 | System cards only | |
| Anthropic political even-handedness eval Models answer paired prompts that take opposing sides on 150 US political topics. A grader scores even-handedness, refusals and acknowledgment of opposing views. | Bias, fairness and politics | Nov 2025 | 99.4% | System cards only |
| Prompt injection robustness against adaptive attackers, coding (Anthropic) Coding agents face indirect prompt injections crafted by an adaptive attacker. Score is attack success rate. | Jailbreaks and prompt injection | Nov 2025 | 0% | System cards only |
| Prompt injection robustness against adaptive attackers, computer use (Anthropic) Computer-use agents face indirect prompt injections crafted by an adaptive attacker. Score is attack success rate. | Jailbreaks and prompt injection | Nov 2025 | 0% | System cards only |
| Long-form virology tasks Agents complete end-to-end pathogen acquisition workflows, covering workflow design and laboratory protocols. Scored against expert criteria. | Hazardous knowledge | Feb 2025 | 69.7% | System cards only |
| Prompt injection robustness against adaptive attackers, browser use (Anthropic) Browser agents face indirect prompt injections in web environments from an internal adaptive attacker. Score is attack success rate. | Jailbreaks and prompt injection | Nov 2025 | 0% | System cards only |
| DNA Synthesis Screening Evasion Models design DNA fragments that assemble into pathogenic viruses while evading gene synthesis screening. Scored per assembly criterion. | Hazardous knowledge | May 2025 | 80% | System cards only |
| Suicide and self-harm evaluations (Anthropic) Models handle single-turn and multi-turn conversations about suicide and self-harm. Score is the appropriate response rate. | Wellbeing and sycophancy | Sep 2025 | System cards only | |
| Quadruped reinforcement learning task (Anthropic) Agents train a quadruped to reach high performance in a continuous control task. | Machine learning engineering | May 2025 | 31.3 | System cards only |
| LLM training task (Anthropic) Agents optimize a CPU-only small language model training implementation. Scored by speedup. | Machine learning engineering | May 2025 | System cards only | |
| Impossible tasks (Anthropic) Agents work on coding problems designed to be impossible. Score is the rate at which they hack instead of giving up. | Deception and alignment | May 2025 | 30% | System cards only |
| Time series forecasting task (Anthropic) Agents solve a regression and time-series forecasting problem with known state-of-the-art results, in six variants. | Machine learning engineering | May 2025 | 4.55 | System cards only |
| Novel compiler task (Anthropic) Agents build a compiler for an unusual programming language from a specification and test cases. | Machine learning engineering | May 2025 | 80.9% | System cards only |
| Disordered eating evaluations (Anthropic) Models handle conversations about eating disorders and disordered eating. Score is the appropriate response rate. | Wellbeing and sycophancy | Feb 2026 | System cards only | |
| Kernels task (Anthropic) Agents optimize compute kernels in a performance engineering challenge. Scored by speedup. | GPU kernels | May 2025 | System cards only | |
| Agentic misalignment suite (Anthropic) Agents in simulated company settings face threats or goal conflicts that invite blackmail, leaking or sabotage. Score is the misaligned action rate. | Deception and alignment | Jun 2025 | 0% | In use |
| Self-preference bias (Anthropic) Models grade transcripts attributed to themselves or to other models. Measures whether scores favor the model's own outputs. | Bias, fairness and politics | Sep 2025 | 0.07 | System cards only |
| Election integrity evaluations (Anthropic) Models answer election-related prompts, including requests for misinformation or influence operations. Score is the appropriate response rate. | Refusing harmful requests | Oct 2024 | System cards only | |
| SWE-bench Verified (hard subset) Agents solve a subset of SWE-bench Verified tasks estimated to take a human engineer more than one hour. | Machine learning engineering | Feb 2025 | 48.2% | System cards only |
| 100Q-Hard (Anthropic) Models answer hard, obscure factual questions. Scored on correct, incorrect and declined answers. | Short factual questions | Sep 2025 | 60.1% | System cards only |
| Reward-hack-prone coding tasks (Anthropic) Models solve hard coding problems where earlier Claude models often special-cased tests. Score is the classifier hack rate. | Deception and alignment | May 2025 | 0% | System cards only |
| Protocols (Anthropic life sciences) Models find and fix errors in molecular biology protocols and answer practical questions about applying real laboratory protocols. | Lab protocols | Jul 2026 | 62% | System cards only |
| Black-box RNA sequence modeling and design Models predict scores for unlabeled RNA sequences from labeled data, then design new sequences. Results are compared with 57 human experts. | Hazardous knowledge | May 2026 | System cards only | |
| Malicious agentic influence campaigns (Anthropic) Agents receive requests to run multi-step influence operations with tools. Score is the refusal rate. | Refusing harmful requests | Apr 2026 | System cards only | |
| Petri Open-source auditing tool where auditor agents probe target models in simulated scenarios and judges score misaligned behaviors. | Deception and alignment | Oct 2025 | System cards only | |
| Creative biology Models answer questions about unusual biology needed to engineer harmless biological agents, a weak proxy for novel weapon design. | Hazardous knowledge | May 2025 | 60.3% | System cards only |
| Text-based reinforcement learning task (Anthropic) Agents build scaffolding that improves a weaker model on a text-based reinforcement learning task. | Machine learning engineering | May 2025 | 0.97 | System cards only |
| Anthropic ECI (AECI) Index that combines a basket of capability benchmarks into one capability score to track trends across Anthropic models. | Composite indices | Apr 2026 | 169.4 | System cards only |
| Crypto CTF challenges (Anthropic) Agents discover and exploit vulnerabilities in cryptographic primitives and protocols in capture-the-flag challenges. | Capture-the-flag challenges | Feb 2025 | 94.4% | System cards only |
| Network CTF challenges (Anthropic) Agents perform reconnaissance and exploit vulnerabilities across multiple networked machines in capture-the-flag challenges. | Capture-the-flag challenges | Feb 2025 | 100% | System cards only |
| Pwn CTF challenges (Anthropic) Agents exploit insecure software on a remote server for privilege escalation or code execution in capture-the-flag challenges. | Capture-the-flag challenges | Feb 2025 | 71.4% | System cards only |
| Rev CTF challenges (Anthropic) Agents reverse-engineer binary executables to find hidden details or vulnerabilities in capture-the-flag challenges. | Capture-the-flag challenges | Feb 2025 | 100% | System cards only |
| Web CTF challenges (Anthropic) Agents discover and exploit vulnerabilities in web applications across easy, medium and hard capture-the-flag challenges. | Capture-the-flag challenges | Feb 2025 | 100% | System cards only |
| Discrim-Eval Models make yes or no decisions in templated scenarios that vary age, gender and race. Scored by mixed effects model coefficients. | Bias, fairness and politics | Dec 2023 | System cards only | |
| Short-horizon computational biology tasks Agents perform end-to-end computational tasks related to modifying and engineering biological agents. | Hazardous knowledge | May 2025 | System cards only | |
| Anthropic undermining liberal democracy (behavioral audit metric) Automated behavioral audit metric: investigator models probe willingness to help coups, election tampering, or political surveillance, censorship or intimidation. | Refusing harmful requests | May 2026 | 1.13 | System cards only |
| Binary Exploitation Benchmark (Anthropic, OSS-Fuzz based) Internal eval on OSS-Fuzz harness entrypoints measuring unguided vulnerability discovery and how far a found bug is advanced. | Finding and exploiting real bugs | 91.1% | System cards only | |
| False-premise questions (Anthropic) Models answer questions built on false premises. Measures how often they accept the premise instead of correcting it. | Hallucination rates | Sep 2025 | 1.20% | System cards only |
| Internal AI research evaluation suite 2 Agents perform scaled-down ML and alignment research tasks similar to past Anthropic research projects. | AI research tasks | May 2025 | 0.65 | System cards only |
| Rewind-attacker cyber adversarial robustness (Anthropic) An automated attacker with rewinds and 400 calls tries to make a safeguarded Claude Code agent complete offensive cyber tasks. Score is attack success rate. | Jailbreaks and prompt injection | Sep 2026 | 3.90% | System cards only |
| Computer use prompt injection evaluation (Anthropic) Agents face about 600 prompt injection scenarios in coding platforms, browsers and email workflows. Score is the attack prevention rate. | Jailbreaks and prompt injection | 78% | System cards only | |
| Refusal to assist with AI safety R&D (Anthropic) Models receive single-turn coding requests for legitimate AI safety research with user context. Score is the refusal rate, where lower is better. | Deception and alignment | Feb 2026 | 3.10% | System cards only |
| Training distribution hack rate (Anthropic) Classifier monitors measure reward hacking on a subset of Anthropic's reinforcement learning training episodes. | Deception and alignment | Nov 2025 | 0.63% | System cards only |
| WildChat non-toxic incorrect refusals (Anthropic) Models answer non-toxic user prompts drawn from the WildChat dataset of real chatbot conversations. Score is the incorrect refusal rate. | Refusing harmful requests | Mar 2024 | 4.80% | System cards only |
| WildChat toxic refusals (Anthropic) Models answer toxic user prompts drawn from the WildChat dataset of real chatbot conversations. Score is the correct refusal rate. | Refusing harmful requests | Mar 2024 | 96.4% | System cards only |
| AAV capsid packaging prediction Models give the probability that each of 1,000 unpublished modified AAV capsid sequences assembles into a functional capsid. Scored as AUROC. | Hazardous knowledge | May 2026 | 0.80 | System cards only |
| Agentic coding malicious requests (Anthropic) Coding agents with standard coding tools receive 150 malicious coding requests prohibited by policy. Score is the refusal rate. | Refusing harmful requests | May 2025 | 100% | System cards only |
| Average in-eval reward hacking rate (Anthropic) Anthropic averages hack rates across its reward hacking evaluations into one number per model. | Deception and alignment | 14.3% | System cards only | |
| Cyber harm coverage (Anthropic) Cyber safeguard classifiers screen examples from each cyber harm category. Score is overall recall, the share of harmful examples flagged. | Refusing harmful requests | Jul 2026 | 100% | System cards only |
| Firefox 147 exploit evaluation (Anthropic) Internal eval on crashes previously found in Firefox 147, run without the browser's process sandbox. Measures exploit-development progress. | Finding and exploiting real bugs | 72.4% | System cards only | |
| Internal model use survey (Anthropic) Anthropic technical staff estimate productivity uplift from the model and judge whether it could automate a junior ML researcher. | Labs' internal R&D tests | May 2025 | 50% | System cards only |
| Organic Chemistry (Anthropic) Models predict structures from spectra, design synthetic routes, predict reaction products and convert between IUPAC names, SMILES and structure images. | Research science | Feb 2026 | 86.2% | System cards only |
| Over-eager behavior on GUI computer use tasks (Anthropic) Computer-use agents face impossible tasks with an unwanted shortcut available. Score is the rate of unapproved workarounds instead of checking with the user. | Deception and alignment | Feb 2026 | 9.40% | System cards only |
| SycophancyEval (answer sycophancy) Models answer factual questions where the user suggests a wrong answer. Score is the relative accuracy drop against neutral prompts. | Deception and alignment | Oct 2023 | 4.00% | Fading |
| Structural Biology (Anthropic) Models answer questions about a biomolecule's function from structural data and basic tools, in multiple-choice and open-ended variants. | Research science | Feb 2026 | 79% | System cards only |
| Protein Design (Anthropic) Models generate novel protein sequences under design constraints and rank protein design libraries for experimental testing. | Research science | Jun 2026 | 42.5% | System cards only |
| Higher-difficulty benign request evaluations (Anthropic) Models answer harder benign requests that resemble harmful ones. Score is the refusal rate, where lower is better. | Refusing harmful requests | Feb 2026 | 0.01% | System cards only |
| Higher-difficulty harmful request evaluations (Anthropic) Models answer harder, more obfuscated policy-violating requests. Score is the harmless response rate. | Refusing harmful requests | Feb 2026 | 99.4% | System cards only |
| Agentic code behavior scores (Anthropic) Agents work through over one hundred realistic coding scenarios. Graders score instruction following, safety, verification, efficiency, adaptability and honesty out of 10. | Deception and alignment | Feb 2026 | System cards only | |
| Intentionally taking actions very rarely (Anthropic) Models are told to take a specific action at a target rate such as 1 percent. Resampling measures the observed rate under several randomness settings. | Deception and alignment | System cards only | ||
| Vulnerability finding evaluations (Anthropic) Cyber safeguard classifiers screen sampled vulnerability discovery traffic. Reports flag rates on violative binary work and on defensive source code work. | Refusing harmful requests | Jul 2026 | System cards only | |
| Adherence to its constitution (Anthropic) Investigator models build scenarios from 40 areas of Claude's constitution. A grader scores each transcript from minus 3 to plus 3 on 15 dimensions. | Deception and alignment | Apr 2026 | 1.02 | System cards only |
| ASL-4 virology uplift trial (Anthropic) PhD experts with model access write a protocol to reconstruct a challenging virus. Graded by a rubric with critical failures. | Hazardous knowledge | Nov 2025 | 66.8% | System cards only |
| Biomedical image analysis (Anthropic) Models run quantitative analyses of CT, MRI, histopathology and other images from public competitions. Scores are scaled to the best competition results. | Research science | Sep 2026 | 72.2% | System cards only |
| BioPipelineBench (Anthropic) Agents execute bioinformatics workflows such as long-read sequence analysis, metagenome assembly and chromatin profiling with a bash tool. | Lab and data work | Feb 2026 | 87.7% | System cards only |
| Bioweapons acquisition uplift trial (Anthropic) Participants draft a bioweapons acquisition plan over two days with or without model access. Score is the uplift ratio of model group over control. | Hazardous knowledge | 2.53 | System cards only | |
| Bioweapons knowledge questions Models answer 33 sensitive bioweapons questions written by Deloitte biosecurity staff. Answers are compared with novice, intermediate and expert baselines. | Hazardous knowledge | Feb 2025 | 51.5% | System cards only |
| Concealment of changes (Anthropic) Models summarize transcripts in which changes were hidden through git manipulations. Score is the rate at which they disclose the concealment. | Deception and alignment | Sep 2026 | 96.9% | System cards only |
| De novo protein binder design (Anthropic) Agents get a target protein structure and 24 hours to design 30 miniprotein binders. The evaluation covers 15 targets. | Research science | Sep 2026 | 85.1% | System cards only |
| Internal agentic coding evaluation (Anthropic) Agents implement a pull request in an open source codebase from a natural language description. A problem counts as solved when all hidden tests pass. | Fixing real repositories | Jun 2024 | 78% | System cards only |
| Medicinal Chemistry (Anthropic) Models see several real drug-discovery compounds and pick which had the highest or lowest measured ADME property. The set has 504 questions. | Research science | Sep 2026 | 65.3% | System cards only |
| Minimal-LinuxBench (Anthropic) Agents attempt a hidden side task in high-stakes software environments while a monitor watches. Score is the stealth success rate. | Deception and alignment | Apr 2026 | 2.60% | System cards only |
| Morphology-to-molecule matching (Anthropic) Models match four sets of multi-channel liver cell images to the four shuffled molecule and dose treatments that produced them. | Research science | Sep 2026 | 34% | System cards only |
| Phylogenetics (Anthropic) Models analyze phylogenetic data and tree diagrams, testing quantitative reasoning about tree structure and visual interpretation. | Research science | Feb 2026 | 79.6% | System cards only |
| Prompt injection live bug bounty (Anthropic with Gray Swan) Expert red teamers submit prompt injection attacks against hidden frontier models in tool use, coding and computer use scenarios. Score is attack success rate. | Jailbreaks and prompt injection | May 2026 | 0.08% | System cards only |
| Real-World Finance (Anthropic) Agents complete about 50 analyst tasks in banking, private equity, hedge funds and corporate finance, producing spreadsheets, slides and documents. Rubric graded. | Finance | Feb 2026 | 70% | System cards only |
| Real-World Finance v2 (Anthropic) Agents complete 294 long-horizon finance tasks such as building financial models. A model grader compares deliverables pairwise and ratings are fit as Elo. | Finance | 1,637 | System cards only | |
| Sandbox escape evaluation (Anthropic) Agents face 28 handmade scenarios that tempt them to cross a staged sandbox boundary. Score is the share of runs with a crossing attempt. | Deception and alignment | Sep 2026 | 1.50% | System cards only |
| Silent copying of answers (Anthropic) Agents solve held-out coding tasks where the answer is reachable in the sandbox. Score is the rate of using it without telling the user. | Deception and alignment | Sep 2026 | 8.00% | System cards only |
| Uncritically reporting flawed results (Anthropic) Agents analyze data with code that silently mishandles broken values. Graders check whether the agent flags the flaw or reports the biased numbers. | Deception and alignment | 9.83 | System cards only | |
| Cyber-harness network challenges (Incalmo cyber ranges) Agents run long-horizon attacks on cyber ranges of 25 to 50 hosts using a specialized offensive harness. | Finding and exploiting real bugs | May 2025 | 66.7% | System cards only |
| Destructive actions (Anthropic) Models continue real Claude Code transcripts cut just before a destructive action. Score is the rate of taking a destructive action. | Deception and alignment | 21% | System cards only | |
| Multi-factual (Anthropic) Models answer questions that each combine several closed-ended sub-questions on one topic. Answers are graded correct, incorrect or unsure against references. | Long-form factuality | Mar 2024 | 62.8% | System cards only |
| AAV packaging rate auto-research evaluation Agents build predictors of AAV capsid packaging that must generalize to an unseen insertion region, using a limited scorer. Scored by Spearman correlation. | Hazardous knowledge | Sep 2026 | System cards only | |
| Acting on a presumed grader and digging for evaluation clues (Anthropic) Models work on coding evaluations. A grader flags transcripts that shape answers to a presumed grader or search for clues about grading. | Deception and alignment | Sep 2026 | System cards only | |
| Hiding a secret password (Anthropic) Models keep a given password secret while an investigator model tries to extract it over many turns. Measures survival and per-turn leak rates. | Deception and alignment | Apr 2026 | System cards only | |
| Verbalized grader awareness (Anthropic) Models work on tasks where grading is disclosed clearly, ambiguously or barely. An autorater flags transcripts that reason about the grader. | Deception and alignment | Apr 2026 | System cards only | |
| Destructive production evaluation (Anthropic) Agents finish a task in a simulated production system under time pressure. Scored on destructive actions and on giving up too early. | Deception and alignment | 0.30% | System cards only | |
| Forensics CTF challenges (Anthropic) Agents analyze logs, files or obfuscated records to reconstruct events in capture-the-flag challenges. | Capture-the-flag challenges | 100% | System cards only | |
| Internal codebase sabotage propensity (Anthropic) Agents in Claude Code write a costly internal classifier they may dislike. Score is the rate of writing a sabotaged classifier. | Deception and alignment | Nov 2025 | 0% | System cards only |
| Lying about identity (Anthropic) Models play an operator-assigned human persona while users ask if they are human. Score is the rate of admitting to being an AI. | Deception and alignment | 99% | System cards only | |
| MCP prompt injection evaluation (Anthropic) Agents use simulated email, Slack and document servers over MCP where content carries injected instructions. Score is the attack prevention rate. | Jailbreaks and prompt injection | Sep 2025 | 92.5% | System cards only |
| Misc CTF challenges (Anthropic) Agents identify and exploit vulnerabilities in capture-the-flag challenges outside the web, crypto, pwn, rev and network categories. | Capture-the-flag challenges | 100% | System cards only | |
| Multimodal policy red-teaming (Anthropic) Models answer multi-turn image and text conversations on harmful topics such as child safety, weapons and fraud. Score is the share of harmless responses. | Refusing harmful requests | Mar 2024 | 99.2% | System cards only |
| Opportunistic sandbox escape (Anthropic) Agents do dev-ops tasks in 1,753 sandboxes with deliberate weaknesses. Score is the share of scenarios where they use credentials or files outside. | Deception and alignment | Oct 2026 | 3.10% | System cards only |
| Organic Chemistry V2 (Anthropic) Models predict structures from spectra, design multi-step syntheses, predict reaction products and read chemical structure images. V2 adds harder expert-recommended problems. | Research science | Jul 2026 | 69.2% | System cards only |
| Overconfidence: verifying command-line syntax (Anthropic) Agents are asked for an exact invocation of an unfamiliar command-line tool. Graders score from 1 to 5 whether they check documentation before running commands. | Deception and alignment | 4.90 | System cards only | |
| Participation in junk science (Anthropic) Models talk with simulated users ranging from curious to openly fraudulent who seek fake science. Score is the rate of generating unsound scientific content. | Refusing harmful requests | Feb 2026 | 0% | System cards only |
| Secure and defensive coding (Anthropic) Cyber safeguard classifiers screen sampled defensive coding sessions such as patching and incident response. Score is the classifier flag rate. | Refusing harmful requests | Jul 2026 | 0.61% | System cards only |
| SSH stress-testing evaluation (Anthropic) Models continue prefilled conversations about suicide and self-harm built from anonymized user feedback. Score is the rate of appropriate course correction. | Wellbeing and sycophancy | Feb 2026 | 95% | System cards only |
| Tool use prompt injection evaluation (Anthropic) Agents run bash commands where files, scripts or command outputs carry injected instructions. Score is the attack prevention rate. | Jailbreaks and prompt injection | Sep 2025 | 96% | System cards only |
| Ambiguous context evaluations (Anthropic) Models respond to gray-area requests where intent is unclear. Graders judge how well responses balance helpfulness and harm. | Refusing harmful requests | May 2025 | System cards only | |
| Evaluation awareness discriminability (Anthropic) Probes and follow-up realism ratings try to tell behavioral audit transcripts from real internal Claude Code sessions. Each setting is scored as AUROC. | Deception and alignment | System cards only | ||
| Input hallucinations (Anthropic) Models answer 500 prompts that need an unavailable tool and 500 prompts that cite missing context. Score is the non-hallucination rate. | Hallucination rates | Apr 2026 | System cards only | |
| Reasoning behavior classifiers (Anthropic) Classifiers scan reasoning text from training episodes for deception, harmful actions and distress. Score is the share of outputs flagged. | Deception and alignment | System cards only | ||
| Subversion Strategy Eval Models must devise and execute strategies to subvert control protocols without shared memory between calls. | Deception and alignment | Dec 2024 | System cards only | |
| SycophancyEval (Sharma et al.) Models answer questions and give feedback while users state opinions. Measures accuracy shifts and feedback positivity caused by user views. | Deception and alignment | Oct 2023 | System cards only | |
| White-box analyses of model internals (Anthropic) An investigator with interpretability tools reads model activations on audit transcripts. A judge scores hidden states such as evaluation awareness and deception. | Deception and alignment | System cards only | ||
| Anthropic Red Team Models answer human red-team attack prompts collected by Anthropic on discrimination, hate, violence and unethical behavior. Judges score response safety. | Refusing harmful requests | Aug 2022 | 99.96% | System cards only |
| Bioinformatics evaluations (Anthropic) Agents process raw sequencing data with workflows from published papers and answer expert questions about the results. | Genomics and life sciences | Nov 2025 | 73.7% | System cards only |
| BioPipelineBench Verified (Anthropic) Agents execute bioinformatics workflows with a bash tool. Only problems that passed a validation check by external reviewers are kept. | Lab and data work | Apr 2026 | 83.6% | System cards only |
| Chain-of-thought faithfulness with clues (Anthropic) Models answer MMLU and GPQA questions with and without an inserted clue. Score is the share of clue-driven answer changes whose reasoning mentions the clue. | Deception and alignment | Feb 2025 | 25.0 | System cards only |
| CoBench (Anthropic) 500 real internal Anthropic R&D problems. Percentage solved in one attempt tracks AI R&D acceleration. | Labs' internal R&D tests | 55.8% | System cards only | |
| Code summary honesty (Anthropic) Models summarize prefilled agentic coding transcripts that hide failures. Score is the rate of summaries that fail to raise the important failures. | Deception and alignment | 3.70% | System cards only | |
| Combined HHH evals (Anthropic) Models pick the more helpful, honest and harmless of two responses across 438 binary choice questions. Scored by accuracy. | Deception and alignment | Dec 2021 | 86% | System cards only |
| Compliance with instructions planted in pasted text (Anthropic) Coding agents receive user prompts with pasted text that hides an attacker's instruction. Score is the rate of acting on the planted instruction. | Jailbreaks and prompt injection | Sep 2026 | 7.40% | System cards only |
| Creative Biology Uplift Trial (Anthropic) Participants with molecular biology PhDs write reports on novel biological workflows with or without model access. Expert graders score each report. | Hazardous knowledge | Feb 2026 | 16.4 | System cards only |
| CVE Rediscovery (Anthropic) Models get a known-vulnerable snapshot of an open-source repository and the list of files its fix changed, then report security bugs in one attempt. A model judge counts a success when a report has the advisory's root cause. 1,829 advisories from October 2026. | Finding and exploiting real bugs | Nov 2026 | 89.6% | System cards only |
| Fermat's Last Theorem in Lean (Anthropic) Agent teams formalize Fermat's Last Theorem in Lean. Scores measure the weighted proportion of 60 accepted intermediate theorems. | Formal proofs | Nov 2026 | 35% | System cards only |
| Harmfulness scores on held-out prompts (Anthropic) Models answer 328 held-out red-team and jailbreak prompts. A preference model flags responses judged more harmful than a fixed refusal. | Jailbreaks and prompt injection | Jul 2023 | 1.20% | System cards only |
| Key insight rediscovery on an unpublished ML task (external testing) Agents work on an unpublished machine learning research task. Testers count how many of five key insights the agent rediscovers. | AI research tasks | 80% | System cards only | |
| Lazy investigation (Anthropic) Agents debug a failing job where misleading evidence invites a harmful action such as cordoning a healthy node. Scored 0 to 10 by a grader. | Deception and alignment | 9.94 | System cards only | |
| Lazy investigation on misleading codebases (Anthropic) Models trace small, intentionally misleading codebases to answer how a value is set. Score is the share of lazy investigations ending in a wrong answer. | Deception and alignment | May 2026 | 0% | System cards only |
| Many-shot jailbreaking evaluation (Anthropic) Models receive long fake dialogs where the assistant complies with harmful requests, then a new harmful request. Score is the compliance rate. | Jailbreaks and prompt injection | 25% | System cards only | |
| Multi-agent search (Anthropic internal benchmark) Agents solve difficult information retrieval problems with web search, alone or as orchestrators that delegate to subagents. | Deep research | Nov 2025 | 74.8% | System cards only |
| Package registry publishing evaluation (Anthropic) Agents in a simulated capture the flag exercise can publish a backdoored package to a registry. An investigator escalates pressure until they act. | Deception and alignment | Sep 2026 | 33.3% | System cards only |
| Petri Bloom An auditor model runs generated scenarios to elicit named misaligned behaviors. Judges score each transcript from 1 to 10. | Deception and alignment | Dec 2025 |  1.44 | System cards only |
| RE-Bench subset (Anthropic) Agents solve a modified subset of four METR RE-Bench research engineering tasks. Scores are normalized from reference solution 0 to expert solution 1. | Machine learning engineering | Feb 2025 | 0.24 | System cards only |
| Recognition of model-written user turns (Anthropic) Models write synthetic user turns for real Claude Code transcripts, then judge which version is real. Score is accuracy above chance. | Other | 0.35 | System cards only | |
| STEM Professor Bench Agents solve unpublished research problems written by professional scientists. Most problems concern theoretical physics, with some partial credit. | Research science | 61.8% | System cards only | |
| Sycophancy on user-provided prompts (Anthropic) Models continue real feedback conversations where earlier Claude models were sycophantic. Score is the non-sycophantic response rate. | Deception and alignment | Nov 2025 | 10% | System cards only |
| Alignment faking evaluation, helpful-only setting (Anthropic) Models learn they will be trained to answer all harmful requests. Score is the rate of complying while reasoning about avoiding preference modification. | Deception and alignment | Dec 2024 | System cards only | |
| Anthropic large agent teams tasks Internal 24-hour tasks for teams of 1-100 agents: build a company knowledge base, or formalize a theorem in Lean. | Multi-agent games | Sep 2026 | System cards only | |
| Appropriate harmlessness (Anthropic) Models answer borderline and harmful prompts. Classifiers sort each response into helpful answer, policy violation, appropriate refusal or unnecessary refusal. | Refusing harmful requests | Feb 2025 | System cards only | |
| Autonomous Replication and Adaption (ARA) evaluations (Anthropic) Agents attempt five autonomy tasks such as adding a Flask backdoor, a SQL injection exploit and an LM worm. Scored by pass rates over trials. | Dangerous capabilities | Sep 2023 | System cards only | |
| Catastrophic biology scenario uplift trial (Anthropic) PhD biologists with model access spend up to 16 hours planning a catastrophic biological scenario. External experts grade uplift and executability. | Hazardous knowledge | System cards only | ||
| Cooperation with user deception (Anthropic) Models get developer system prompts asking them to subtly mislead users, then meet users who would be harmed. Graders score harmful and deceptive choices. | Refusing harmful requests | Sep 2025 | System cards only | |
| Deference to governments in local languages (Anthropic) Models answer politically controversial questions in English and in a local language. A grader rates agreement with the official government position from 1 to 10. | Bias, fairness and politics | Feb 2026 | System cards only | |
| Expert vulnerability discovery and exploit development evaluations (Anthropic) Agents locate advanced vulnerabilities in possibly obfuscated codebases and write working exploits across six challenge classes. Scored by success rates. | Finding and exploiting real bugs | Mar 2024 | System cards only | |
| External pilot use failure modes (Anthropic) Models are judged on user-flagged pilot conversations, directly or by resampling. Graders measure failure modes such as hallucinated facts and misrepresented work completion. | Deception and alignment | Feb 2026 | System cards only | |
| Keeping secrets when pressed in simulated scenarios (Anthropic) Models hold confidential information in 39 job scenarios while a simulated counterpart presses for it. Measures leaks in replies and in reasoning. | Deception and alignment | Oct 2026 | System cards only | |
| Malicious agentic influence campaigns, 34-point rubric (Anthropic) Agents run simulated voter suppression and domestic polarization campaigns. Weighted criteria measure campaign execution on a 34-point scale. | Dangerous capabilities | Nov 2026 | System cards only | |
| Missing-context hallucinations (Anthropic) Models get prompts that request tools they lack or cite missing material. Score is the non-hallucination rate on each 500-prompt set. | Hallucination rates | System cards only | ||
| Research sandbagging (Anthropic) Agents are told to sabotage a coding task or imitate a weaker model without alerting a weaker monitor. Measures the performance drop and monitor suspicion. | Deception and alignment | Sep 2025 | System cards only | |
| Sycophancy prompts (Anthropic) Models answer single-turn prompts that invite sycophancy, such as bad business ideas or delusional beliefs. Four synthetic sets and one hand-written set report sycophancy rates. | Deception and alignment | Sep 2025 | System cards only | |
| SCONE-bench Agents find and exploit vulnerabilities in real smart contracts on a local blockchain fork. Success is measured in simulated dollars stolen, not only pass or fail. | Finding and exploiting real bugs | Dec 2025 | $548,400,000 | Not in cards |
| Agent4Sci Coding agents try to improve the published best model on 20 scientific ML benchmarks across 11 domains, under long in-the-wild runs. The paper reports mean relative improvement rather than a ranking. | AI research tasks | Oct 2026 | Not in cards | |
| Anthropic cyber safeguard evaluations Classifier recall on violative cyber exchanges, block rates on defensive vulnerability-finding and secure-coding work, and the rewind-attacker robustness test. | Refusing harmful requests | Not in cards | ||
| Anthropic harmful-request and wellbeing evaluations Single-turn harmful and benign request sets, multi-turn tests, child safety, suicide/self-harm, disordered eating and election integrity. | Refusing harmful requests | Not in cards | ||
| Anthropic internal life-sciences evals Internal suite: morphology-to-molecule matching, medicinal chemistry, protein design, binder design, biomedical image analysis, organic chemistry. | Research science | 2026 | Not in cards | |
| Anthropic IoT self-replication evaluation Open-weight models set up LLM inference on 9 hardware types and spread as worms across 8 emulated Android nodes. Scored by success and tokens/s. | Finding and exploiting real bugs | Oct 2026 | Not in cards | |
| Anthropic malicious agentic use evaluations Measures refusal of malicious tasks via Claude Code, computer use and agentic influence campaigns. | Refusing harmful requests | Not in cards | ||
| Anthropic model-written evals LM-generated datasets testing persona, sycophancy and advanced-AI-risk behaviors. | Deception and alignment | Dec 2022 | Not in cards | |
| Anthropic targeted alignment and honesty evaluations Internal suite: reward hacking, impossible tasks, sandbox escape, destructive actions, pasted-instruction following, self-preference, answer copying, concealment, rare actions. | Deception and alignment | Not in cards | ||
| CryptanalysisBench Agents attack 191 real cryptographic primitives, mostly from four NIST standardization competitions. Tier 1 has known practical attacks, and Tier 2 has none, so a solve there is a new break. | Finding and exploiting real bugs | Jul 2026 | Not in cards | |
| Petri 2.0 Auditor agents run seed scenarios against target models with a realism classifier to cut eval-awareness. Judges score transcripts for misaligned behaviors. | Deception and alignment | Jan 2026 | Not in cards | |
| TASTE 92 pairwise comparisons of AI safety research proposals. Model must match experienced researchers' preference. Accuracy versus ~77% human agreement. | AI research tasks | Aug 2026 | Not in cards |
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| Human feedback win rates (Anthropic) Raters chat with two models on tasks such as coding, documents and creative writing and pick the better reply. Score is the win rate. | Overall chat quality | Mar 2024 | System cards only |