Claude 3.5 Sonnet ranks first on 6 of the 139 benchmarks with five or more models scored.
Closed weights Mid-size
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| EQ-Bench v2 | 86.4 | #1 of 85 | Official leaderboard | |
| MixEval | 89.9% | #1 of 34 | Official leaderboard | |
| BIG-Bench Hard | 93.1% | #3 of 73 | Lab model card | |
| ThaiExam | 75.1% | #1 of 24 | Official leaderboard | |
| MMLU | 90.4% | #11 of 131 | Lab model card | |
| MGSM | 91.6% | #6 of 57 | Lab model card | |
| WildBench | 54.5 | #6 of 47 | Official leaderboard | |
| FACTS Grounding | 83.3% | #5 of 34 | Official leaderboard | |
| Chinese SimpleQA | 53.5% | #5 of 27 | Official leaderboard | |
| Arena-Hard-Auto | 79.3% | #8 of 38 | Official leaderboard | |
| BullshitBench | 52.7% | #24 of 107 | Official leaderboard | |
| SEAL Precise Instruction Following | 86.0 | #5 of 18 | Official leaderboard | |
| MMBench-CN v1.1 | 79.3% | #53 of 178 | Official leaderboard | |
| MMLU-Pro | 76.1% | #69 of 212 | Official leaderboard | |
| Needle In A Haystack | 99.7% | #3 of 9 | Lab model card | |
| Honesty human feedback win rate (Anthropic) | 66% | #2 of 6 | Lab model card | |
| DROP | 87.1% | #14 of 41 | Lab model card | |
| JudgeBench | 64.3% | #4 of 11 | Official leaderboard | |
| RewardBench | 84.2% | #19 of 48 | Official leaderboard | |
| ECLeKTic | 20% | #15 of 35 | Official leaderboard | |
| BullshitBench V2 | 46% | #58 of 129 | Official leaderboard | |
| AidanBench | 1439.0 | #6 of 13 | Official leaderboard | |
| MARS-Bench | 43.2% | #7 of 15 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 4.60% | #81 of 148 | Official leaderboard | |
| LMArena (Elo) | 1,343 | #184 of 335 | Official leaderboard | |
| Longform Creative Writing (EQ-Bench) | 58.4 | #67 of 119 | Official leaderboard | |
| Creative Writing v3 | 64.2 | #73 of 120 | Official leaderboard | |
| RewardBench 2 | 64.7% | #27 of 42 | Official leaderboard | |
| EQ-Bench 3 | 1,102 | #52 of 78 | Official leaderboard | |
| SEAL Multilingual | 992 | #13 of 19 | Official leaderboard | |
| MultiLoKo | 23.2% | #5 of 7 | Official leaderboard | |
| WritingBench | 53.8 | #36 of 45 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 96.4% | #4 of 92 | Lab model card | |
| GSM1k | 96.6% | #1 of 15 | Official leaderboard | |
| OlympicArena | 39.2% | #2 of 25 | Official leaderboard | |
| KOR-Bench | 59% | #3 of 33 | Official leaderboard | |
| ZebraLogic | 33.4% | #8 of 46 | Official leaderboard | |
| MATH | 71.1% | #44 of 99 | Lab model card | |
| ForecastBench | 0.13 | #40 of 82 | Official leaderboard | |
| MATH Level 5 | 51.7% | #49 of 94 | Epoch AI run | |
| Omni-MATH | 26.2% | #5 of 9 | Official leaderboard | |
| GPQA Diamond | 54% | #149 of 229 | Epoch AI run | |
| DTBench | 67.8% | #120 of 180 | Official leaderboard | |
| GSM8K Platinum Initial 268 | 98.1% | #14 of 19 | Official leaderboard | |
| SimpleBench | 27.5% | #86 of 106 | Official leaderboard | |
| 3SAT-Search | 12.9% | #9 of 11 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 6.53% | #175 of 212 | Epoch AI run | |
| FrontierMath Tier 4 | 0% | #46 of 55 | Epoch AI run | |
| FrontierMath | 2.07% | #48 of 55 | Official leaderboard | |
| MATH-500 | 72.4% | #35 of 38 | Official leaderboard | |
| AIME 2025 | 3.33% | #53 of 53 | Official leaderboard | |
| HMMT 2025 | 1.67% | #53 of 53 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval | 92% | #9 of 101 | Lab model card | |
| Aider-Edit | 77.4% | #4 of 34 | Official leaderboard | |
| BigCodeBench | 52.7% | #9 of 69 | Official leaderboard | |
| SWE-bench Lite | 51.3% | #2 of 15 | Official leaderboard | |
| SciCode | 4.60% | #3 of 18 | Official leaderboard | |
| MBPP+ | 74.3% | #6 of 35 | Official leaderboard | |
| BigCodeBench-Hard | 30.4% | #19 of 96 | Official leaderboard | |
| MBPP | 89.4% | #7 of 35 | Official leaderboard | |
| MHPP | 45.2% | #5 of 23 | Official leaderboard | |
| HumanEval+ | 81.7% | #11 of 46 | Official leaderboard | |
| GitTaskBench | 40.7% | #3 of 10 | Official leaderboard | |
| SWE-bench | 24.9% | #3 of 9 | Official leaderboard | |
| SEAL Coding | 1,079 | #9 of 26 | Official leaderboard | |
| SWE-bench Verified | 63.4% | #26 of 60 | Official leaderboard | |
| Aider Polyglot | 51.6% | #21 of 46 | Official leaderboard | |
| METR Time Horizon | 20 min | #20 of 31 | Official leaderboard | |
| WeirdML | 40% | #77 of 112 | Official leaderboard | |
| METR Time Horizon 1.1 | 11 min | #18 of 26 | Official leaderboard | |
| BIRD-CRITIC | 21.5% | #7 of 9 | Official leaderboard | |
| Spider 2.0 | 19% | #18 of 23 | Official leaderboard | |
| SWE-bench Multimodal | 12.2% | #8 of 9 | Official leaderboard | |
| MLE-bench | 7.56% | #11 of 12 | Official leaderboard | |
| Internal agentic coding evaluation (Anthropic) | 64% | #3 of 3 | Lab model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GroundUI | 17.3% | #3 of 11 | Official leaderboard | |
| BALROG | 32.6% | #12 of 33 | Official leaderboard | |
| GAIA | 65.1% | #24 of 64 | Official leaderboard | |
| CRMArena | 41.8% | #4 of 9 | Official leaderboard | |
| ToolComp | 59.4% | #17 of 32 | Official leaderboard | |
| AceBench | 68.9% | #9 of 16 | Official leaderboard | |
| LisanBench | 365.3 | #75 of 133 | Official leaderboard | |
| LLM Chess | -134 | #85 of 123 | Official leaderboard | |
| lmgame-Bench | 13.5 | #19 of 23 | Official leaderboard | |
| Online-Mind2Web | 29% | #5 of 5 | Official leaderboard | |
| tau-bench Airline | 36% | #2 of 2 | Official leaderboard | |
| tau-bench Retail | 62.6% | #2 of 2 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MedQA | 82.5% | #4 of 71 | Official leaderboard | |
| MortgageTax | 64.1% | #45 of 87 | Official leaderboard | |
| MedXpertQA-MM | 33.2% | #6 of 9 | Official leaderboard | |
| TaxEval v2 | 70.2% | #87 of 129 | Official leaderboard | |
| FailureSensorIQ | 58.6% | #17 of 24 | Official leaderboard | |
| CorpFin v2 | 53.6% | #90 of 119 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SEED-Bench-2-Plus | 71.7% | #6 of 136 | Official leaderboard | |
| SGP-Bench | 67.4% | #1 of 22 | Official leaderboard | |
| CharXiv | 60.2% | #4 of 77 | Official leaderboard | |
| ChartQA | 90.8% | #2 of 37 | Lab model card | |
| AI2D | 94.7% | #4 of 52 | Lab model card | |
| CoreCognition | 61.9% | #14 of 119 | Official leaderboard | |
| MEGA-Bench | 52.1% | #3 of 22 | Official leaderboard | |
| DynaMath | 64.8% | #2 of 11 | Official leaderboard | |
| Image2Struct | 73.7 | #4 of 18 | Official leaderboard | |
| VCR (Visual Caption Restoration) | 35.6% | #10 of 42 | Official leaderboard | |
| MMStar | 62.2% | #46 of 173 | Official leaderboard | |
| DocVQA | 95.2% | #10 of 36 | Lab model card | |
| MMMU | 68.3% | #38 of 136 | Official leaderboard | |
| MicroVQA | 51.7% | #6 of 19 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 54.1% | #53 of 152 | Official leaderboard | |
| VisualPuzzles | 42.4% | #11 of 29 | Official leaderboard | |
| MMBench | 79.7% | #59 of 155 | Official leaderboard | |
| MMBench-EN v1.1 | 77.7% | #73 of 178 | Official leaderboard | |
| MMMU-Pro | 51.5% | #25 of 57 | Official leaderboard | |
| TOMATO | 27.8% | #11 of 22 | Official leaderboard | |
| MATH-Vision (MathVision) | 38% | #63 of 105 | Official leaderboard | |
| VHELM | 48.2 | #25 of 41 | Official leaderboard | |
| Video-MME | 62.9% | #17 of 26 | Official leaderboard | |
| VISTA | 38.4% | #36 of 54 | Official leaderboard | |
| GeoBench | 62% | #33 of 48 | Official leaderboard | |
| Vision Arena | 1,146 | #101 of 136 | Official leaderboard | |
| MathVista | 67.7% | #52 of 70 | Lab model card | |
| ZeroBench | 2.00% | #63 of 76 | Official leaderboard | |
| VPCT | 33% | #25 of 27 | VPCT leaderboard, via Epoch AI |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SimpleSafetyTests | 100% | #4 of 75 | Official leaderboard | |
| HELM Safety | 97.7 | #8 of 75 | Official leaderboard | |
| Situational Awareness Dataset (SAD) | 57.1% | #2 of 17 | Official leaderboard | |
| Anthropic Red Team | 99.8% | #9 of 75 | Official leaderboard | |
| XSTest | 1.70% | #2 of 16 | Lab model card | |
| AIR-Bench 2024 | 85.9% | #11 of 82 | Official leaderboard | |
| FORTRESS | 13% | #10 of 73 | Official leaderboard | |
| NYU CTF Bench | 22% | #1 of 7 | Official leaderboard | |
| SORRY-Bench | 14.3% | #6 of 23 | Official leaderboard | |
| LAB-Bench CloningScenarios | 28% | #1 of 3 | Official leaderboard | |
| LAB-Bench FigQA | 46% | #1 of 3 | Official leaderboard | |
| WildChat toxic refusals (Anthropic) | 96.4% | #1 of 3 | Lab model card | |
| BaxBench | 36% | #12 of 34 | Official leaderboard | |
| Spiral-Bench v1.2 | 42.6 | #9 of 21 | Official leaderboard | |
| Cybench | 17.5% | #7 of 16 | Official leaderboard | |
| OR-Bench | 43.8% | #13 of 28 | Official leaderboard | |
| BrowserART | 22% | #3 of 6 | Official leaderboard | |
| Harmlessness human feedback win rate (Anthropic) | 52% | #2 of 4 | Lab model card | |
| SEAL Adversarial Robustness | 16.0 | #5 of 8 | Official leaderboard | |
| LAB-Bench | 35.1% | #2 of 3 | Official leaderboard | |
| LAB-Bench ProtocolQA | 48% | #2 of 3 | Official leaderboard | |
| Human Pathogen Capabilities Test | 42.7% | #63 of 86 | Official leaderboard | |
| Spiral-Bench v1.0 | 47.7 | #10 of 13 | Official leaderboard | |
| AgentDojo | 33.9% | #19 of 20 | Official leaderboard | |
| LAB-Bench LitQA2 | 6.00% | #3 of 3 | Official leaderboard | |
| WildChat non-toxic incorrect refusals (Anthropic) | 11% | #3 of 3 | Lab model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HELM Lite | 88.5 | #4 of 71 | Official leaderboard | |
| OpenVLM Leaderboard | 67.9 | #45 of 173 | Official leaderboard |
The Self-Reported Index fits one score to the 14 capability results Anthropic reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 129.3 to 134.6. How we compute it.
Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.
Anthropic's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)
The benchmarks the lab chose to headline, with the scores it gave.
Every result in the longer documents the lab published with it. 8 of them aren't in the launch post.