Benchmark Almanac

Claude 3.7 Sonnet

Claude 3.7 Sonnet ranks first on 16 of the 197 benchmarks with five or more models scored.

Anthropic Released February 24th, 2025 Family Claude 3.7 The lab's model card Compare with others
211
Benchmarks with a score
16
Ranked first, out of five or more models
36
In the top three
Anthropic Feb 2025

Claude 3.7 Sonnet

Closed weights Mid-size

141.0 #118 of 256
Epoch Capabilities Index
Self-Reported Index 140.6 #172 of 404
  • Humanity's Last Exam 8.04% #34 /47
  • SWE-bench Verified 61% #28 /32
  • MultiChallenge 51.6% #28 /42

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
MLogiQA 67.1% #1 of 17 Official leaderboard
MARS-Bench 62.3% #1 of 15 Official leaderboard
KoBALT 61% #1 of 12 Official leaderboard
IFEval 93.2% #16 of 135 Lab model card
BuzzBench 68.2 #3 of 20 Official leaderboard
Fiction.LiveBench 83.3% #9 of 57 Fiction.live leaderboard, via Epoch AI
AraGen 78.2% #7 of 42 Official leaderboard
MMLU-Pro 84% #36 of 212 Official leaderboard
Video SimpleQA 36.2% #6 of 27 Official leaderboard
BullshitBench 52.7% #25 of 107 Official leaderboard
RewardBench 2 75.4% #10 of 42 Official leaderboard
MMBench-CN v1.1 79.5% #51 of 178 Official leaderboard
Arena-Hard v2.0 59.8% #7 of 23 Official leaderboard
ECLeKTic 23.3% #11 of 35 Official leaderboard
LLM Confabulation Leaderboard 14.7% #21 of 58 Official leaderboard
MMMLU 86.1% #26 of 71 Lab model card
Phare 69.5% #23 of 62 Official leaderboard
FaithJudge 16.1% #14 of 37 Official leaderboard
BullshitBench V2 49% #52 of 129 Official leaderboard
FACTS Grounding 78.8% #14 of 34 Official leaderboard
LMArena (Elo) 1,388 #145 of 335 Official leaderboard
WritingBench 73.4 #20 of 45 Official leaderboard
SnorkelSpatial 21.5% #13 of 27 Official leaderboard
Vectara Hallucination Leaderboard (HHEM) 4.40% #76 of 148 Official leaderboard
MultiNRC 27.8% #20 of 38 Official leaderboard
Longform Creative Writing (EQ-Bench) 57.7 #71 of 119 Official leaderboard
Creative Writing v3 64.7 #72 of 120 Official leaderboard
EQ-Bench 3 1,128 #48 of 78 Official leaderboard
MultiChallenge 51.6% #28 of 42 Official leaderboard
Global MMLU Lite 80.8% #31 of 43 Official leaderboard
Humanity's Last Exam 8.04% #34 of 47 Official leaderboard
HUMAINE 20 #43 of 50 Official leaderboard

Math and science

BenchmarkScoreRankSource
GSM8K-Platinum 99.8% #1 of 8 Official leaderboard
GSM8K Platinum Initial 268 99.6% #3 of 19 Official leaderboard
MATH Level 5 91.2% #15 of 94 Epoch AI run
TSP-Opt 47.7% #2 of 11 Official leaderboard
PhyX 42.2% #2 of 10 Official leaderboard
ForecastBench 0.13 #17 of 82 Official leaderboard
Misguided Attention 56.1% #10 of 45 Official leaderboard
CORE-Bench 37.8% #6 of 17 Official leaderboard
GPQA Main 60.8% #24 of 60 Official leaderboard
GPQA Diamond 79.7% #97 of 229 Epoch AI run
MATH-500 96.2% #32 of 70 Lab model card
PolyMath 33.5% #18 of 38 Official leaderboard
AIME 2024 80% #42 of 88 Lab model card
3SAT-Search 61.8% #6 of 11 Official leaderboard
SnorkelGraph 50% #14 of 24 Official leaderboard
OTIS Mock AIME 2024-2025 57.8% #128 of 212 Epoch AI run
SimpleBench 46.4% #65 of 106 Official leaderboard
SnorkelSequences 47.6% #16 of 25 Official leaderboard
USAMO 2025 3.65% #7 of 10 Official leaderboard
SciArena 966 #24 of 34 Official leaderboard
IneqMath 2.00% #26 of 34 Official leaderboard
FrontierMath 4.14% #43 of 55 Official leaderboard
ARC-AGI 28.6% #77 of 95 Official leaderboard
CMT-Benchmark 6.00% #9 of 11 Official leaderboard
PHYBench 15.3% #23 of 28 Official leaderboard
PutnamBench 0% #10 of 12 Official leaderboard
Putnam-like 67.9% #5 of 6 Official leaderboard
ARC-AGI-2 0.90% #84 of 98 Official leaderboard
UGMathBench 73.7% #6 of 7 Official leaderboard
HMMT 2025 31.7% #46 of 53 Official leaderboard
AIME 2025 49.2% #47 of 53 Official leaderboard
SMT 2025 56.6% #39 of 40 Official leaderboard
BRUMO 2025 65.8% #42 of 42 Official leaderboard
CombiBench 2.00% #4 of 4 Official leaderboard

Coding

BenchmarkScoreRankSource
BigCodeBench-Hard 35.8% #1 of 96 Official leaderboard
ArtifactsBench (Gemini 2.5 Pro 0506 judge) 52.2% #1 of 18 Official leaderboard
GitTaskBench 48.1% #1 of 10 Official leaderboard
Multi-SWE-bench 9.25% #2 of 11 Official leaderboard
SWE-bench 33.8% #2 of 9 Official leaderboard
Aider Polyglot 64.9% #11 of 46 Official leaderboard
SWE-bench Lite 48% #4 of 15 Official leaderboard
METR Time Horizon 55 min #11 of 31 Official leaderboard
Multi-SWE-bench mini 18.3% #2 of 5 Official leaderboard
SWT-bench 52.5% #3 of 7 Official leaderboard
SWE-bench Multimodal 31.3% #4 of 9 Official leaderboard
ArtifactsBench 51.3% #9 of 20 Official leaderboard
Multi-SWE-bench flash 21.3% #2 of 4 Official leaderboard
SWE-Perf 2.26% #1 of 2 Official leaderboard
METR Time Horizon 1.1 60 min #14 of 26 Official leaderboard
DesignArena Website 1,186 #83 of 151 Official leaderboard
DesignArena Data Visualization 1,194 #69 of 118 Official leaderboard
DesignArena Code 1,167 #85 of 143 Official leaderboard
DesignArena Game Dev 1,128 #93 of 143 Official leaderboard
Spider 2.0 24.5% #15 of 23 Official leaderboard
DesignArena UI Component 1,142 #91 of 139 Official leaderboard
LiveSQLBench 21.7% #24 of 36 Official leaderboard
DesignArena 3D Design 1,077 #100 of 136 Official leaderboard
DesignArena SVG 1,078 #76 of 103 Official leaderboard
SWE-bench-Live 17.7% #9 of 12 Official leaderboard
GSO-Bench 3.80% #26 of 30 Official leaderboard
SWE-bench Verified 61% #28 of 32 Epoch AI run
SWE-fficiency 2.44% #17 of 17 Official leaderboard
SWE-bench Verified (hard subset) 23% #9 of 9 Lab model card
CATArena Gomoku 748 #5 of 5 Official leaderboard
CATArena Texas Holdem 935 #5 of 5 Official leaderboard
METR data deduplication 13.3% #3 of 3 Lab model card
RE-Bench subset (Anthropic) 0.24 #1 of 1 Lab model card

Agents and tools

BenchmarkScoreRankSource
Galileo Agent Leaderboard 0.95 #1 of 29 Official leaderboard
C3-Bench 11.3% #4 of 30 Official leaderboard
Elimination Game 5 #9 of 56 Official leaderboard
tau-bench 69.8% #4 of 23 Lab model card
UI-Vision Element Grounding 8.27% #1 of 5 Official leaderboard
GAIA 74.8% #13 of 64 Official leaderboard
ToolComp 65.3% #7 of 32 Official leaderboard
AgentBench FC 53.2% #3 of 12 Official leaderboard
lmgame-Bench 35.6 #7 of 23 Official leaderboard
SnakeBench 27 #58 of 188 Official leaderboard
BrowseComp-ZH 17.7% #3 of 9 Official leaderboard
ScienceBoard 15.8% #6 of 15 Official leaderboard
VideoGameBench Lite 1.60% #2 of 5 Official leaderboard
Berkeley Function Calling Leaderboard (BFCL) 58.6% #27 of 67 Official leaderboard
Step Race 3 #25 of 59 Official leaderboard
BrowseComp-VL 11.2% #2 of 4 Official leaderboard
LisanBench 498.3 #67 of 133 Official leaderboard
LLM Chess -1 #65 of 123 Official leaderboard
MCP-Universe 24.2% #18 of 30 Official leaderboard
Online-Mind2Web 56.3% #3 of 5 Official leaderboard
OSWorld 35.8% #5 of 8 OS World Website, via Epoch AI
Orak 6.30 #4 of 6 Official leaderboard
OSWorld-Verified 35.8% #23 of 33 Official leaderboard
tau2-bench 61.8% #7 of 10 Official leaderboard
SnorkelWordle 68% #15 of 19 Official leaderboard
VideoGameBench 0% #4 of 5 Official leaderboard
UI-Vision Layout Grounding 17.6% #2 of 2 Official leaderboard

Professional work

BenchmarkScoreRankSource
LEXam 62.9% #2 of 30 Official leaderboard
MortgageTax 68.7% #11 of 87 Official leaderboard
Finance Reasoning (Snorkel) 51.9% #3 of 21 Official leaderboard
xbench-Recruitment 61.4 #2 of 8 Official leaderboard
Vals Finance Agent v1 44.1% #12 of 47 Official leaderboard
TheAgentCompany 30.9% #2 of 7 Official leaderboard
xbench-Marketing 47.6 #2 of 7 Official leaderboard
TaxEval v2 74% #41 of 129 Official leaderboard
FinQA (Snorkel OpenEnv environment) 77.9% #7 of 22 Official leaderboard
SnorkelFinance 77.9% #7 of 22 Official leaderboard
SnorkelUnderwrite 74.6% #10 of 27 Official leaderboard
CadEval 54% #6 of 13 Official leaderboard
LegalBench 82.5% #68 of 138 Official leaderboard
MedHELM 52.9 #7 of 13 Official leaderboard
CorpFin v2 60.4% #68 of 119 Official leaderboard
HealthBench 34.6% #6 of 8 Official leaderboard
HealthBench Hard 1.52% #6 of 8 Official leaderboard

Vision, audio and robotics

BenchmarkScoreRankSource
CharXiv 64.2% #3 of 77 Official leaderboard
VRBench 68.2% #1 of 19 Official leaderboard
EmbodiedBench EB-ALFRED 67.7% #1 of 17 Official leaderboard
EmbodiedBench EB-Habitat 58.7% #1 of 17 Official leaderboard
EmbodiedBench EB-Manipulation 28.5% #1 of 17 Official leaderboard
MM-Vet 70% #21 of 164 Official leaderboard
CoreCognition 61.4% #17 of 119 Official leaderboard
MMMU 75% #21 of 136 Official leaderboard
MMStar 65.1% #33 of 173 Official leaderboard
VisualPuzzles 48.3% #6 of 29 Official leaderboard
SEED-Bench-2-Plus 67.6% #33 of 136 Official leaderboard
MMBench 81.4% #44 of 155 Official leaderboard
MMSI-Bench 30.2% #8 of 28 Official leaderboard
SpatialViz-Bench 33.9% #4 of 14 Official leaderboard
VISTA 48.2% #16 of 54 Official leaderboard
MMBench-EN v1.1 79.8% #55 of 178 Official leaderboard
SpaCE-10 46.2% #12 of 36 Official leaderboard
All-Angles Bench 50% #6 of 18 Official leaderboard
OCR-Reasoning 35.8% #5 of 15 Official leaderboard
CCBench (MMBench Chinese Culture) 55.1% #51 of 152 Official leaderboard
EmbodiedBench EB-Navigation 45% #6 of 17 Official leaderboard
MATH-Vision (MathVision) 58.6% #39 of 105 Official leaderboard
A-Bench 70.7% #66 of 116 Official leaderboard
ZeroBench 6.00% #44 of 76 Official leaderboard
Vision Arena 1,193 #84 of 136 Official leaderboard
GeoBench 68% #30 of 48 Official leaderboard
VPCT 39% #17 of 27 VPCT leaderboard, via Epoch AI
MME-Reasoning 34.1% #7 of 11 Official leaderboard
VLMsAreBiased 20.9% #4 of 5 Official leaderboard
Point-Bench 22.2% #14 of 17 Official leaderboard
ERQA 14.3% #17 of 17 Official leaderboard

Safety and security

BenchmarkScoreRankSource
Video-SafetyBench 1.30 #1 of 15 Official leaderboard
SimpleSafetyTests 100% #8 of 75 Official leaderboard
AIRTBench 46.9% #1 of 9 Official leaderboard
Gray Swan Agent Red Teaming (ART) benchmark 1.47% #1 of 8 Official leaderboard
Anthropic Red Team 99.7% #11 of 75 Official leaderboard
BaxBench 46.4% #5 of 34 Official leaderboard
AIR-Bench 2024 81.8% #21 of 82 Official leaderboard
NYU CTF Bench 18.5% #2 of 7 Official leaderboard
AgentDojo 7.31% #6 of 20 Official leaderboard
Lakera B3 (Backbone Breaker Benchmark) 71.1% #11 of 35 Official leaderboard
Bioweapons knowledge questions 51.5% #1 of 3 Lab model card
HELM Safety 94.5 #26 of 75 Official leaderboard
Cybench 20% #6 of 16 Official leaderboard
Computer use prompt injection evaluation (Anthropic) 74% #2 of 5 Lab model card
AutoBaxBench 32.1% #5 of 11 Official leaderboard
BBQ (Bias Benchmark for QA) 84% #15 of 25 Lab model card
Crypto CTF challenges (Anthropic) 75% #4 of 6 Lab model card
Web CTF challenges (Anthropic) 87.5% #4 of 6 Lab model card
AutoPatchBench 10.6% #2 of 3 Official leaderboard
Bioweapons acquisition uplift trial (Anthropic) 2.10 #2 of 3 Lab model card
FORTRESS 38% #49 of 73 Official leaderboard
Virology Capabilities Test (VCT) 30.8% #5 of 7 Official leaderboard
CyberGym 14.5% #27 of 34 Official leaderboard
Pwn CTF challenges (Anthropic) 38.5% #5 of 6 Lab model card
SCONE-bench $3,500,000 #7 of 7 Official leaderboard
Network CTF challenges (Anthropic) 0% #6 of 6 Lab model card
Rev CTF challenges (Anthropic) 0% #6 of 6 Lab model card
Chain-of-thought faithfulness with clues (Anthropic) 25.0 #1 of 1 Official leaderboard
Cyber-harness network challenges (Incalmo cyber ranges) 66.7% #1 of 1 Lab model card
Long-form virology tasks 69.7% #1 of 1 Lab model card
SEC-bench 18% #1 of 1 Official leaderboard

Other

BenchmarkScoreRankSource
OpenVLM Leaderboard 70.1 #36 of 173 Official leaderboard
CAIS Risk Index 46.0 #25 of 67 Official leaderboard
HELM Capabilities 67.4 #26 of 60 Official leaderboard
TutorBench 46.5% #19 of 20 Official leaderboard
CAIS Text Capabilities Index 8.00 #66 of 68 Official leaderboard
CAIS Vision Capabilities Index 33.9 #52 of 53 Official leaderboard

Its lab's numbers put it at 140.6, in line with its ECI

The Self-Reported Index fits one score to the 9 capability results Anthropic reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 137.1 to 144.2. How we compute it.

Its lab's numbers next to 6 leaderboards

Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.

Benchmark Lab says Board says Gap What differs
Cybench 44.1% Cybench, succeeded at least once on 15/34 challenges tested (pass@k over up to 30 trials). Best-of-k curve on a 30-challenge subset has no printed values (7.3.2, Figure 25) 20% Leaderboard +24 pts Anthropic counts challenges solved at least once across up to 30 trials, while the board counts eight solved challenges in its run.
IFEval 93.2% Instruction-following IFEval (64K extended thinking) 83.4% Leaderboard, strict, share of instructions followed +9.8 pts
GPQA Diamond 84.8% Graduate-level reasoning GPQA Diamond (64K extended thinking, parallel test-time compute) 79.7% Epoch AI run, 64k thinking +5.1 pts
MATH-500 96.2% Math problem-solving MATH 500 (64K extended thinking) 91.6% Leaderboard, Vals AI harness +4.6 pts
SWE-bench Verified 70.3% Agentic coding SWE-bench Verified (custom scaffold, reduced subset) 66.4% Leaderboard, Aime-coder v1 +3.9 pts
MMMU 75% Visual reasoning MMMU (validation) (64K extended thinking) 75% Leaderboard Same

Anthropic's models over time

Anthropic's models in color, against every other model on the Epoch Capabilities Index, by release date.

50 100 150 2023 2024 2025 2026 2027 Claude Fable 5.2 Claude Haiku 5.5 Claude Sonnet 4.6 Claude Haiku 4.5 Claude 3.7 Sonnet Claude 3.5 Haiku Claude 3.5 Sonnet (October 2024) 50 100 150 2023 2024 2025 2026 2027 Claude Fable 5.2 Claude Haiku 5.5

Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Claude 3.7 Sonnet and Claude Code (launch post) Feb 24, 2025

Its system card

Every result in the longer documents the lab published with it. 28 of them aren't in the launch post.

Claude 3.7 Sonnet System Card Feb 24, 2025

Safety

Robustness

Bias

Alignment

Monitorability

Bio and chem risk

Cyber risk

AI R&D risk