- Agentic coding Terminal-Bench 4.0 70.6
- Agentic coding FrontierCode 1.1 (Main) (Xhigh) 52.1
- Agentic coding CursorBench 4.0 55.5
- Knowledge work GDPval-AA v2.1 1844
- Knowledge work AA-Briefcase v1.1 1811
- Multidisciplinary reasoning Humanity's Last Exam (with tools) 64.5
- Computer use OSWorld 2.1 (partial) 80.1
- Visual chart recognition Chartography (no tools) 61.6
- SWE-Bench Pro 81.3
- SWE-Bench Multilingual 90.3
- SWE-Bench Multimodal 54.3
- HealthBench Professional 69.2
- AutomationBench 44.7
Claude Sonnet 5.5
Claude Sonnet 5.5 ranks first on 10 of the 287 benchmarks with five or more models scored.
Claude Sonnet 5.5
Closed weights Mid-size
- Humanity's Last Exam 46.1% #4 /47
- AutomationBench 44.8% #5 /42
- Terminal-Bench 4.0 61.8% #2 /40
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Humanity's Last Exam | 46.1% | #4 of 47 | Official leaderboard | |
| LMArena (Elo) | 1,472 | #39 of 335 | Official leaderboard | |
| HANDBOOK.md | 47.7% | #2 of 16 | Official leaderboard | |
| Joke Arena | 1,550 | #4 of 25 | Official leaderboard | |
| AA-LCR Extended | 73.8% | #8 of 49 | Official leaderboard | |
| ComplexConstraints | 67.2% | #4 of 15 | Official leaderboard | |
| BullshitBench V2 | 60% | #35 of 129 | Official leaderboard | |
| Corp Speak Arena | 1,562 | #7 of 22 | Official leaderboard | |
| SimpleQA Verified | 44.8% | #19 of 56 | Official leaderboard | |
| Global MMLU | 92.1% | #5 of 14 | Lab model card | |
| Global MMLU Lite | 92.6% | #16 of 43 | Official leaderboard | |
| Extended NYT Connections | 80.5% | #41 of 103 | Official leaderboard | |
| HLE-Diamond | 43.3% | #8 of 20 | Official leaderboard | |
| FACTS Multimodal | 50.4% | #10 of 23 | Official leaderboard | |
| MILU | 91.6% | #5 of 11 | Lab model card | |
| FACTS Grounding v2 | 62.5% | #16 of 25 | Official leaderboard | |
| FACTS Benchmark Suite | 58.7% | #18 of 23 | Official leaderboard | |
| Opening Lines Arena | 1,425 | #12 of 15 | Official leaderboard | |
| FACTS Parametric V2 | 35.1% | #22 of 27 | Official leaderboard | |
| ELI5 Arena | 1,407 | #16 of 18 | Official leaderboard | |
| StudyArena Text | 1,170 | #79 of 85 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GPQA Diamond | 95.6% | #4 of 229 | Epoch AI run | |
| OTIS Mock AIME 2024-2025 | 100% | #10 of 212 | Epoch AI run | |
| DTBench | 97.3% | #10 of 180 | Official leaderboard | |
| ProofBench v1.1 | 100% | #3 of 52 | Official leaderboard | |
| BioMysteryBench | 81.1% | #2 of 30 | Official leaderboard | |
| LatchBio Capabilities v1.0 | 54.2% | #2 of 27 | Official leaderboard | |
| FrontierMath Tiers 1-3 v2 | 88.8% | #10 of 86 | Official leaderboard | |
| MysteryMechanism | 49.1% | #4 of 31 | Official leaderboard | |
| ARC-AGI | 96.5% | #15 of 95 | Official leaderboard | |
| FrontierMath Tier 4 v2 | 80.5% | #11 of 68 | Official leaderboard | |
| SimpleBench | 75.9% | #18 of 106 | Official leaderboard | |
| ARC-AGI-2 | 83.9% | #18 of 98 | Official leaderboard | |
| scBench (LatchBio SingleCellBench) | 60.3% | #6 of 26 | Official leaderboard | |
| Riemann-bench | 76.8% | #6 of 20 | Official leaderboard | |
| FrontierMath Erdős | 2.90% | #5 of 15 | Official leaderboard | |
| Terminal-Bench-Science | 59.9% | #4 of 12 | Lab model card | |
| Biomedical image analysis (Anthropic) | 72.2% | #1 of 3 | Lab model card | |
| Medicinal Chemistry (Anthropic) | 65.3% | #1 of 3 | Lab model card | |
| Prophet Arena | 85.8 | #9 of 24 | Official leaderboard | |
| SpatialBench (LatchBio) | 72.5% | #3 of 7 | Lab model card | |
| ArXivMath | 86.8% | #4 of 9 | Lab model card | |
| De novo protein binder design (Anthropic) | 82.3% | #3 of 3 | Lab model card | |
| Morphology-to-molecule matching (Anthropic) | 25% | #3 of 3 | Lab model card |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Code Migration | 69.8% | #1 of 78 | Official leaderboard | |
| Vibe Code Bench | 92.4% | #2 of 120 | Official leaderboard | |
| DesignArena Data Visualization | 1,360 | #2 of 118 | Official leaderboard | |
| DesignArena UI Component | 1,386 | #3 of 139 | Official leaderboard | |
| Code Arena | 1,773 | #3 of 128 | Official leaderboard | |
| MirrorCode | 97.8% | #1 of 39 | Epoch AI run | |
| SWE Atlas Codebase QnA Extended | 44.7% | #1 of 38 | Official leaderboard | |
| WebDev Arena | 1,786 | #3 of 105 | arena.ai, via Epoch AI | |
| APEX-SWE | 66.4% | #2 of 65 | Official leaderboard | |
| Code Migration v1.1 | 86.9% | #1 of 30 | Official leaderboard | |
| Image-to-WebDev Arena | 1,744 | #2 of 55 | Official leaderboard | |
| SWE-Bench Pro | 81.3% | #3 of 81 | Lab model card | |
| SWE-bench Multilingual | 90.3% | #3 of 73 | Lab model card | |
| DesignArena Game Dev | 1,391 | #8 of 143 | Official leaderboard | |
| Android Bench 2.0 | 40% | #1 of 17 | Official leaderboard | |
| DesignArena 3D Design | 1,401 | #9 of 136 | Official leaderboard | |
| DesignArena Code | 1,347 | #11 of 143 | Official leaderboard | |
| DesignArena SVG | 1,303 | #9 of 103 | Official leaderboard | |
| DesignArena React Native | 1,261 | #5 of 56 | Official leaderboard | |
| ALE-Bench | 2,448 | #12 of 126 | Official leaderboard | |
| DesignArena Website | 1,311 | #15 of 151 | Official leaderboard | |
| DesignArena Android (Kotlin) | 1,260 | #5 of 46 | Official leaderboard | |
| sudo L7 | 50.6% | #2 of 18 | Official leaderboard | |
| DesignArena Agentic Overall | 1,280 | #9 of 74 | Official leaderboard | |
| DAYJOB: Data Science | 35.3% | #2 of 15 | Official leaderboard | |
| FrontierCode 2.0 | 42.2% | #5 of 32 | Official leaderboard | |
| Brokk Power Ranking | 26.9 | #3 of 19 | Official leaderboard | |
| FrontierSWE | 61.9% | #4 of 25 | Official leaderboard | |
| FDE-Gym | 0.49 | #2 of 12 | Official leaderboard | |
| DesignArena Frontend Web App | 1,277 | #9 of 53 | Official leaderboard | |
| CursorBench | 55.5% | #4 of 21 | cursor.com, via Epoch AI | |
| CursorBench 4.0 | 55.5% | #4 of 21 | Official leaderboard | |
| IOI v2 (Vals) | 83.1% | #9 of 45 | Official leaderboard | |
| SWE-Marathon v1.1 | 51.3% | #4 of 20 | Official leaderboard | |
| Real-SWE 1.1 | 51.2% | #3 of 15 | Official leaderboard | |
| FrontierCS 2 | 50% | #4 of 19 | Official leaderboard | |
| DesignArena Image to HTML | 1,232 | #11 of 51 | Official leaderboard | |
| SWE Atlas v1.1 | 68.3% | #7 of 28 | Official leaderboard | |
| DesignArena Image to Frontend | 1,249 | #8 of 31 | Official leaderboard | |
| OpenDesign Arena | 86.8 | #6 of 21 | Official leaderboard | |
| WeirdML v3 | 19.9% | #8 of 27 | Official leaderboard | |
| DAXBench v2 | 84.6% | #14 of 47 | Official leaderboard | |
| DeepSWE | 71% | #19 of 56 | Lab model card | |
| OWE-Bench | 39% | #5 of 14 | Official leaderboard | |
| FrontierCode 1.1 | 52.1% | #5 of 13 | Lab model card | |
| DesignArena Blender | 1,268 | #5 of 12 | Official leaderboard | |
| ViBench 2.0 | 19.1% | #4 of 9 | Official leaderboard | |
| SWE-Together | 62.4% | #10 of 22 | Official leaderboard | |
| SWE-bench Verified Extended | 57.3% | #21 of 44 | Official leaderboard | |
| SWE-bench Multimodal | 54.3% | #6 of 12 | Lab model card | |
| RSI Bench | 35.2 | #4 of 8 | Official leaderboard | |
| FrontierCS 1.1 | 53.9 | #3 of 6 | Official leaderboard | |
| SWE-Bench Pro V2 | 95.8% | #10 of 18 | Official leaderboard | |
| ProgramBench tests passed | 79.7% | #7 of 10 | Lab model card | |
| SWE-Bench Pro V2 Hard | 70.6% | #14 of 18 | Official leaderboard | |
| Chart Arena | 1,457 | #12 of 15 | Official leaderboard | |
| Floorplan Arena | 1,257 | #11 of 11 | Official leaderboard | |
| ReviewBench | 63.1% | #4 of 4 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BrowseComp Extended | 73% | #1 of 42 | Official leaderboard | |
| Terminal-Bench 4.0 | 61.8% | #2 of 40 | Official leaderboard | |
| LisanBench | 13787.0 | #9 of 133 | Official leaderboard | |
| Terminal-Bench 4.1 | 63.9% | #3 of 43 | Official leaderboard | |
| Agent Arena | 15.9% | #4 of 56 | Official leaderboard | |
| Mystery Game Puzzles | 65% | #7 of 84 | Epoch AI run | |
| o11y-bench | 90.6% | #2 of 22 | Official leaderboard | |
| DuelLab GameBench 2 | 91.4 | #6 of 57 | Official leaderboard | |
| AutomationBench | 44.8% | #5 of 42 | Official leaderboard | |
| Chess Puzzles | 59% | #12 of 93 | Epoch AI run | |
| LLM Chess | 1,195 | #16 of 123 | Official leaderboard | |
| Terminal-Bench 2.1 Extended | 45.5% | #6 of 44 | Official leaderboard | |
| Cresta-Bench | 78% | #2 of 14 | Official leaderboard | |
| Game Arena Bargaining | 243 | #6 of 39 | Official leaderboard | |
| Game Arena Nine Men's Morris | 912 | #7 of 39 | Official leaderboard | |
| Game Arena Dots and Boxes | 1,055 | #8 of 40 | Official leaderboard | |
| ARC-AGI-3 | 72.4% | #6 of 27 | Official leaderboard | |
| Game Arena Go | 1,442 | #9 of 39 | Official leaderboard | |
| Toolathlon-Verified | 77.8% | #5 of 21 | Lab model card | |
| Collaborative Intelligence Benchmark | 67.9% | #5 of 18 | Official leaderboard | |
| OSWorld 2.1 | 80.1% | #2 of 7 | Lab model card | |
| RuneBench | 5.52 | #24 of 76 | Official leaderboard | |
| Hermes Bench | 73.5% | #6 of 17 | Official leaderboard | |
| Game Arena Checkers | 980 | #17 of 41 | Official leaderboard | |
| Game Arena Dark Hex | 281 | #17 of 41 | Official leaderboard | |
| DRACO | 87% | #5 of 12 | Lab model card | |
| Game Arena Four in a Row | 412 | #20 of 43 | Official leaderboard | |
| Game Arena Reversi | 540 | #20 of 39 | Official leaderboard | |
| Game Arena Word Art | 604 | #20 of 39 | Official leaderboard | |
| Game Arena Chess | 783 | #22 of 42 | Official leaderboard | |
| Kaggle Game Arena | 322 | #22 of 42 | Official leaderboard | |
| BU Bench V2.1 | 72.7 | #9 of 16 | Official leaderboard | |
| CivBench | 1,497 | #11 of 19 | Official leaderboard | |
| Game Arena Lines of Action | 338 | #24 of 39 | Official leaderboard | |
| Game Arena Ultimate Tic-Tac-Toe | 245 | #24 of 39 | Official leaderboard | |
| Game Arena Clobber | 170 | #26 of 39 | Official leaderboard | |
| StarSkirmish Bench | 25.8% | #6 of 9 | Official leaderboard | |
| Game Arena Word Association | 126 | #29 of 40 | Official leaderboard | |
| WANDR | 70% | #3 of 4 | Lab model card | |
| SpireClimb | 1.33% | #8 of 9 | Epoch AI run | |
| SpireClimb StS2 A10 | 2.67% | #8 of 9 | Official leaderboard | |
| Game Arena Coin Game | 200 | #35 of 39 | Official leaderboard | |
| Web Context (Nimble) | 67.2% | #10 of 11 | Official leaderboard | |
| SpireClimb StS1 A20 Heart | 0% | #9 of 9 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Vals Index v2 | 69.2% | #2 of 65 | Official leaderboard | |
| MedScribe | 91.1% | #3 of 97 | Official leaderboard | |
| HealthBench Professional | 69.2% | #1 of 26 | Lab model card | |
| Excel Modeling Benchmark | 75.7% | #3 of 76 | Official leaderboard | |
| Vals Index v2.1 | 67% | #2 of 49 | Official leaderboard | |
| Tuesday Work Index | 75.8 | #2 of 44 | Official leaderboard | |
| GDPval Extended | 7.00% | #2 of 43 | Official leaderboard | |
| DAYJOB: Finance | 40.5% | #2 of 39 | Official leaderboard | |
| DAYJOB: Healthcare | 34% | #2 of 37 | Official leaderboard | |
| Vals Index v2.2 | 70.8% | #2 of 37 | Official leaderboard | |
| DAYJOB: Accounting | 35.2% | #1 of 17 | Official leaderboard | |
| HealthBench | 65.4% | #2 of 31 | Lab model card | |
| APEX-Agents | 75.5% | #4 of 61 | Collected by Epoch AI, source not named | |
| DAYJOB: Tax | 10.8% | #1 of 15 | Official leaderboard | |
| EnterpriseBench: CoreCraft | 70.8% | #4 of 51 | Official leaderboard | |
| APEX-Agents 1.1 | 83.7% | #6 of 65 | Official leaderboard | |
| Excel Modeling Benchmark v1.1 | 75.9% | #4 of 42 | Official leaderboard | |
| Vending-Bench 2 | $11,165 | #8 of 67 | Official leaderboard | |
| PolicyBench | 91.7% | #6 of 50 | Official leaderboard | |
| MedCode | 52.9% | #12 of 96 | Official leaderboard | |
| Vals Finance Agent v2 | 58.1% | #11 of 80 | Official leaderboard | |
| Tax Agent Bench | 42% | #12 of 84 | Official leaderboard | |
| DesignArena Spreadsheet | 1,295 | #5 of 35 | Official leaderboard | |
| Vals Finance Agent v2.1 | 67.3% | #6 of 40 | Official leaderboard | |
| Vals Finance Index | 64.5% | #6 of 40 | Official leaderboard | |
| GDP.xlsx | 29.1% | #3 of 20 | Official leaderboard | |
| RealmPathology | 77.3% | #2 of 13 | Official leaderboard | |
| Legal Research Bench | 48.1% | #15 of 93 | Official leaderboard | |
| EEBench | 67.2% | #5 of 30 | Official leaderboard | |
| Physical-Engineering's Last Exam | 0.45 | #3 of 18 | Official leaderboard | |
| DeepFinance | 54.6% | #4 of 23 | Official leaderboard | |
| MissionReport | 63.8% | #2 of 10 | Official leaderboard | |
| APEX-Accounting | 64.5% | #10 of 49 | Official leaderboard | |
| FailureSensorIQ | 67.4% | #5 of 24 | Official leaderboard | |
| YC-Bench | $1,741,315 | #13 of 59 | Official leaderboard | |
| RCM-Bench-1 | 91% | #5 of 20 | Official leaderboard | |
| AA-Briefcase v1.1 | 1,811 | #2 of 8 | Lab model card | |
| DesignArena PDF | 1,238 | #10 of 37 | Official leaderboard | |
| Immigration Bench | 69% | #7 of 25 | Official leaderboard | |
| Public Benefits Bench | 67.2% | #15 of 50 | Official leaderboard | |
| GDPval-AA v2.1 | 1,844 | #3 of 10 | Lab model card | |
| DAYJOB: Law | 27% | #5 of 16 | Official leaderboard | |
| Clipping Bench | $886 | #6 of 19 | Official leaderboard | |
| BenchCAD | 74.7% | #4 of 11 | Lab model card | |
| micro1 Medical Index | 70% | #3 of 8 | Official leaderboard | |
| Thrive Holdings helpdesk ticket resolution benchmark | 51.9% | #6 of 15 | Official leaderboard | |
| TaxCalcBench v2 | 50% | #14 of 32 | Official leaderboard | |
| GDPx | 8.22% | #9 of 20 | Official leaderboard | |
| OfficeQA | 76.9% | #7 of 14 | Lab model card | |
| Legora BAR | 102.0 | #9 of 17 | Official leaderboard | |
| MedXpertQA MM Extended | 55.6% | #22 of 41 | Official leaderboard | |
| Harvey LAB (Legal Agent Benchmark) | 2.92% | #45 of 80 | Official leaderboard | |
| BenchCAD 2.0 | 41% | #3 of 5 | Official leaderboard | |
| BenchCAD 2.1 | 60.9% | #3 of 5 | Official leaderboard | |
| PharmaBench (micro1) | 11.4% | #8 of 13 | Official leaderboard | |
| OfficeQA Pro | 65.6% | #10 of 16 | Lab model card | |
| Harvey LAB v1.1 (HLAB-Vals-1.1) | 10.4% | #24 of 38 | Official leaderboard | |
| Quant-Arena | 40.9 | #6 of 9 | Official leaderboard | |
| PhysicianBench | 63.2% | #2 of 3 | Lab model card | |
| Vals Legal Index | 68% | #27 of 38 | Official leaderboard | |
| HW-Bench | 20% | #11 of 15 | Official leaderboard | |
| Short Form Bench | $2,355 | #7 of 9 | Official leaderboard | |
| APEX-1 | 53.9% | #64 of 71 | Official leaderboard | |
| Company Agent Bench | 68.7% | #6 of 6 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| DesignArena Code to Video | 1,318 | #2 of 29 | Official leaderboard | |
| ExtractBench | 94.7% | #2 of 27 | Official leaderboard | |
| MineBench | 1,977 | #8 of 75 | Official leaderboard | |
| Code2Video-Compass | 1.93 | #2 of 15 | Official leaderboard | |
| BVB | 72.3 | #8 of 45 | Official leaderboard | |
| CharXiv Extended | 84.2% | #7 of 38 | Official leaderboard | |
| vCX Benchmark | 91.2% | #8 of 42 | Official leaderboard | |
| Furniture Assembly Benchmark | 75% | #7 of 35 | Official leaderboard | |
| MMMU-Pro Extended | 44.4% | #8 of 38 | Official leaderboard | |
| GDP.pdf | 51.2% | #4 of 19 | Official leaderboard | |
| Blueprint-Bench 2 | 39.3% | #9 of 38 | Official leaderboard | |
| Vision Arena | 1,263 | #36 of 136 | Official leaderboard | |
| ParseBench | 69.2% | #13 of 44 | Official leaderboard | |
| ShadowBench | 48.8 | #2 of 6 | Official leaderboard | |
| Code2Video Arena | 1,542 | #5 of 13 | Official leaderboard | |
| ExtractBench Pro | 32% | #3 of 7 | Official leaderboard | |
| VibeVideoBench | 3.88 | #4 of 9 | Official leaderboard | |
| SplatBench | 5.70 | #5 of 8 | Official leaderboard | |
| Dual-Arm Sim | 26.0 | #7 of 11 | Official leaderboard | |
| OmniParseBench | 88.7 | #6 of 9 | Official leaderboard | |
| Chartography | 61.6% | #10 of 14 | Lab model card |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| World-Class Bio | 66.5% | #3 of 74 | Official leaderboard | |
| Human Pathogen Capabilities Test | 77.2% | #4 of 86 | Official leaderboard | |
| BioSecBench-Function | 53.5% | #2 of 27 | Official leaderboard | |
| scBench-Long | 31.8% | #2 of 27 | Official leaderboard | |
| VariantBench | 51.7% | #2 of 27 | Official leaderboard | |
| Prompt injection robustness against adaptive attackers, browser use (Anthropic) | 0% | #1 of 13 | Lab model card | |
| BioTIER Refuse | 95.7% | #6 of 69 | Official leaderboard | |
| OncologyBench | 42.7% | #3 of 30 | Official leaderboard | |
| BioSecBench-Surveillance | 54.6% | #3 of 27 | Official leaderboard | |
| EpiBench | 30.2% | #3 of 27 | Official leaderboard | |
| MetagenomicsBench | 60% | #3 of 26 | Official leaderboard | |
| Molecular Biology Capabilities Test | 58.8% | #11 of 90 | Official leaderboard | |
| TxBench-Preclinical-Pharmacology | 64.3% | #4 of 30 | Official leaderboard | |
| DeepsecBench | 37.5 | #4 of 29 | Official leaderboard | |
| VCT-v2 | 52.2% | #4 of 25 | Official leaderboard | |
| BioSimBench | 0.16 | #2 of 12 | Official leaderboard | |
| nocot-bench (NCRI) | 95.1 | #31 of 185 | Official leaderboard | |
| SAFE-Teen Urgent safety | 23% | #3 of 17 | Official leaderboard | |
| SHADE-Arena | 22.3% | #3 of 16 | Lab model card | |
| SRE-Bench | 30.1% | #7 of 35 | Official leaderboard | |
| TxBench-Antibody-Discovery | 57.3% | #6 of 30 | Official leaderboard | |
| SpatialBench-Long | 36.5% | #6 of 27 | Official leaderboard | |
| TxBench-Oligo-Discovery | 51.9% | #7 of 30 | Official leaderboard | |
| DistressBench | 74.6% | #12 of 42 | Official leaderboard | |
| Prompt injection robustness against adaptive attackers, computer use (Anthropic) | 0.07% | #4 of 14 | Lab model card | |
| VIALS 2 | 41% | #4 of 14 | Official leaderboard | |
| Virology Capabilities Test (VCT) | 58% | #9 of 31 | Lab model card | |
| SAFE-Teen Relationship boundaries | 37% | #5 of 17 | Official leaderboard | |
| Single-turn benign request evaluations (Anthropic) | 0.02% | #6 of 19 | Lab model card | |
| Binary Exploitation Benchmark (Anthropic, OSS-Fuzz based) | 85.8% | #2 of 6 | Lab model card | |
| Silent copying of answers (Anthropic) | 8.00% | #1 of 3 | Lab model card | |
| CyScenarioBench | 46.1% | #6 of 17 | Lab model card | |
| ExploitBench | 80% | #8 of 21 | Lab model card | |
| ExploitGym | 26.6% | #6 of 15 | Lab model card | |
| SAFE-Teen Health advice | 43% | #7 of 17 | Official leaderboard | |
| WellnessBench | 57% | #5 of 12 | Official leaderboard | |
| Anthropic political even-handedness eval | 97.9% | #6 of 14 | Lab model card | |
| MASK | 94.5% | #9 of 19 | Lab model card | |
| Self-preference bias (Anthropic) | 0.16 | #2 of 4 | Lab model card | |
| SecIT Bench | 75.1% | #14 of 26 | Official leaderboard | |
| Gray Swan IPI Arena benchmark | 0.40% | #6 of 11 | Lab model card | |
| Malicious use of Claude Code (Anthropic) | 85.2% | #10 of 17 | Lab model card | |
| Suicide Memory Bench | 0.65 | #8 of 12 | Official leaderboard | |
| Anthropic undermining liberal democracy (behavioral audit metric) | 1.21 | #4 of 6 | Lab model card | |
| Concealment of changes (Anthropic) | 96.2% | #2 of 3 | Lab model card | |
| Sandbox escape evaluation (Anthropic) | 4.60% | #2 of 3 | Lab model card | |
| Prompt injection robustness against adaptive attackers, coding (Anthropic) | 3.01% | #10 of 14 | Lab model card | |
| CyberBench v1.1 (Vals) | 59.6% | #33 of 44 | Official leaderboard | |
| BBQ (Bias Benchmark for QA) | 80% | #19 of 25 | Lab model card | |
| Biopharma Bench | 49.3% | #10 of 13 | Official leaderboard | |
| Anthropic automated behavioral audit | 2.87 | #5 of 6 | Lab model card | |
| LinuxArena | 36.2% | #5 of 6 | Lab model card | |
| Single-turn harmful request evaluations (Anthropic) | 95.6% | #16 of 19 | Lab model card | |
| BioTIER Permit | 78.5% | #65 of 69 | Official leaderboard | |
| Malicious computer use (Anthropic) | 79.5% | #14 of 14 | Lab model card | |
| Rewind-attacker cyber adversarial robustness (Anthropic) | 21% | #6 of 6 | Lab model card | |
| Cyber harm coverage (Anthropic) | 99.4% | #5 of 5 | Lab model card |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LMCA | 68.3% | #2 of 147 | Official leaderboard | |
| ACCoRD | 85.7 | #3 of 152 | Official leaderboard | |
| Conceptual Reasoning Index | 77.1 | #3 of 151 | Official leaderboard | |
| SAGE (Vals) | 51.8% | #11 of 81 | Official leaderboard | |
| Political Science LLM Benchmark | 68.9% | #6 of 42 | Official leaderboard | |
| Vals Science Index | 59.1% | #4 of 25 | Official leaderboard | |
| Hermes Index | 53.1 | #4 of 17 | Official leaderboard | |
| LiveBench | 77% | #9 of 31 | Official leaderboard | |
| Anthropic ECI (AECI) | 167.9 | #2 of 6 | Lab model card | |
| SAGE v1.1 (Vals) | 88% | #10 of 29 | Official leaderboard |
Its lab's numbers put it at 168.0, 1.4 points above its ECI
The Self-Reported Index fits one score to the 23 capability results Anthropic reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 165.8 to 170.2. How we compute it.
- Global MMLU 92.1
- MILU 91.6
- SWE-bench Multilingual 90.3
- DRACO 87
- ArXivMath 86.8
- SWE-Bench Pro 81.3
- ProgramBench tests passed 79.7
- Toolathlon-Verified 77.8
- OfficeQA 76.9
- BenchCAD 74.7
- DeepSWE 71
- Terminal-Bench 4.0 70.6
- HealthBench Professional 69.2
- OfficeQA Pro 65.6
- HealthBench 65.4
- Humanity's Last Exam 64.5
- FrontierSWE 61.9
- Chartography 61.6
- Terminal-Bench-Science 59.9
- SWE-bench Multimodal 54.3
- FrontierCode 1.1 52.1
- AutomationBench 44.7
- Harvey LAB (Legal Agent Benchmark) 11.7
Its lab's numbers next to 7 leaderboards
Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.
| Benchmark | Lab says | Board says | Gap |
|---|---|---|---|
| Humanity's Last Exam | 64.5% Multidisciplinary reasoning Humanity's Last Exam (with tools) | 46.1% Leaderboard, xhigh | +18 pts |
| Harvey LAB (Legal Agent Benchmark) | 11.7% Legal Agent Benchmark held-out set, all-pass rate at high effort, mean criterion pass 92.1% (max effort 10% and 93.1%) (8.14.2) | 2.92% Leaderboard | +8.8 pts |
| Terminal-Bench 4.0 | 70.6% Agentic coding Terminal-Bench 4.0 | 61.8% Leaderboard, max, Claude Code | +8.8 pts |
| scBench (LatchBio SingleCellBench) | 59.1% LatchBio SingleCellBench (8.17.2, Figure 8.17.A) | 60.3% Leaderboard, published v1, Pi | −1.2 pts |
| AutomationBench | 44.7% AutomationBench | 44.8% Leaderboard, max, default fallbacks | Same |
| FrontierSWE | 61.9% FrontierSWE v2, mean of five trials, Proximal harness at max effort (8.7) | 61.9% Leaderboard, mean@5, proximus | Same |
| CursorBench 4.0 | 55.5% Agentic coding CursorBench 4.0 | 55.5% Leaderboard, max, Cursor | Same |
Anthropic's models over time
Anthropic's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 64 of them aren't in the launch post.
Capabilities
- SWE-Bench Pro 81.3
- SWE-bench Multilingual 90.3
- SWE-bench Multimodal 54.3
- FrontierCode 1.1 46.2
- Humanity's Last Exam 56.9
- OSWorld 2.1 80.1
- GDPval-AA v2.1 1844
- AA-Briefcase v1.1 1811
- AutomationBench 44.7
- DeepSWE 71
- Terminal-Bench 4.0 70.6
- Terminal-Bench-Science 59.9
- FrontierSWE 61.9
- CursorBench 4.0 55.5
- ArXivMath 86.8
- ProgramBench tests passed 79.7
- DRACO 87
- WANDR 70
- Chartography 61.6
- BenchCAD 74.7
- OfficeQA 76.9
- OfficeQA Pro 65.6
- Harvey LAB (Legal Agent Benchmark) 11.7
- Toolathlon-Verified 77.8
- Global MMLU 92.1
- MILU 91.6
- SpatialBench (LatchBio) 72.5
- scBench (LatchBio SingleCellBench) 59.1
- Morphology-to-molecule matching (Anthropic) 25
- Medicinal Chemistry (Anthropic) 65.3
- Protein Design (Anthropic)
- De novo protein binder design (Anthropic) 82.3
- Biomedical image analysis (Anthropic) 72.2
Health
- HealthBench Professional 69.2
- HealthBench 65.4
- PhysicianBench 63.2
Hallucination
Safety
- Single-turn harmful request evaluations (Anthropic) 95.6
- Single-turn benign request evaluations (Anthropic) 0.02
- Multi-turn testing (Anthropic)
- Child safety evaluations (Anthropic)
- Suicide and self-harm evaluations (Anthropic)
- Disordered eating evaluations (Anthropic)
- Election integrity evaluations (Anthropic)
- Malicious use of Claude Code (Anthropic) 85.2
- Malicious computer use (Anthropic) 79.5
Robustness
- Rewind-attacker cyber adversarial robustness (Anthropic) 21
- Gray Swan IPI Arena benchmark 0.4
- Prompt injection robustness against adaptive attackers, coding (Anthropic) 3.01
- Prompt injection robustness against adaptive attackers, computer use (Anthropic) 0.07
- Prompt injection robustness against adaptive attackers, browser use (Anthropic) 0
Alignment
Monitorability
Bio and chem risk
Cyber risk
AI R&D risk
- Anthropic ECI (AECI) 167.9