DeepSeek-R1

DeepSeek-R1 ranks first on 3 of the 74 benchmarks with five or more models scored.

DeepSeek Released January 20th, 2025 Family DeepSeek-R1 The lab's model card Compare with others

Benchmarks with a score

Ranked first, out of five or more models

In the top three

DeepSeek Jan 2025

DeepSeek-R1

Open weights Flagship

138.6 #129 of 256

Epoch Capabilities Index

Self-Reported Index 136.2 #217 of 404

GPQA Diamond 71.7% #115 /229
AMO-Bench 10.9% #16 /21
LiveCodeBench v5 65.9% #16 /47

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

Benchmark	Score	Rank	Source
MMLU	90.8%	#7 of 131	Lab model card
AlpacaEval 2.0	87.6%	#1 of 16	Lab model card
DROP	92.2%	#3 of 41	Lab model card
SEAL Precise Instruction Following	87.8	#2 of 18	Official leaderboard
SuperGPQA	61.8%	#7 of 59	Official leaderboard
LongBench v2	58.3%	#4 of 33	Official leaderboard
CLUEWSC	92.8%	#1 of 8	Lab model card
Arena-Hard-Auto	92.3%	#5 of 39	Lab model card
BBEH micro average	34.9%	#2 of 11	Official leaderboard
LLM Confabulation Leaderboard	12.7%	#11 of 58	Official leaderboard
C-Eval	91.8%	#9 of 46	Lab model card
MMLU-Pro	84%	#42 of 167	Lab model card
Fiction.LiveBench	69.4%	#17 of 57	Fiction.live leaderboard, via Epoch AI
MMLU-Redux	92.9%	#16 of 45	Lab model card
BIG-Bench Extra Hard	6.80%	#4 of 11	Official leaderboard
SimpleQA	30.1%	#23 of 60	Lab model card
Arena-Hard v2.0	58%	#9 of 23	Official leaderboard
LMArena (Elo)	1,398	#134 of 335	Official leaderboard
AgentIF	57.9%	#5 of 12	Official leaderboard
WritingBench	72.1	#23 of 45	Official leaderboard
FRAMES	82.5%	#5 of 9	Lab model card
MultiNRC	24.3%	#24 of 38	Official leaderboard
IFEval	83.3%	#94 of 135	Lab model card
Chinese SimpleQA	63.7%	#12 of 13	Lab model card
Vectara Hallucination Leaderboard (HHEM)	14.3%	#137 of 148	Official leaderboard
WideSearch	0.80%	#6 of 6	Official leaderboard

Math and science

Benchmark	Score	Rank	Source
ZebraLogic	78.7%	#3 of 46	Official leaderboard
PolyMath	47%	#5 of 38	Official leaderboard
U-MATH	80.7%	#4 of 30	Official leaderboard
MATH Level 5	93%	#14 of 94	Epoch AI run
MATH-500	97.3%	#17 of 70	Lab model card
3SAT-Search	80.7%	#3 of 11	Official leaderboard
TSP-Opt	46.6%	#3 of 11	Official leaderboard
Misguided Attention	55.1%	#13 of 45	Official leaderboard
USAMO 2025	4.76%	#4 of 10	Official leaderboard
AIME 2024	79.8%	#43 of 88	Lab model card
GPQA Diamond	71.7%	#115 of 229	Epoch AI run
CNMO 2024	78.8%	#4 of 7	Lab model card
OTIS Mock AIME 2024-2025	53.3%	#133 of 212	Epoch AI run
AIME 2025	70%	#38 of 53	Official leaderboard
AMO-Bench	10.9%	#16 of 21	Official leaderboard
HMMT 2025	41.7%	#41 of 53	Official leaderboard
SimpleBench	30.9%	#83 of 106	Official leaderboard
ARC-AGI-2	1.30%	#80 of 98	Official leaderboard
BRUMO 2025	80.8%	#36 of 42	Official leaderboard
ARC-AGI	15.8%	#83 of 95	Official leaderboard
SMT 2025	67%	#36 of 40	Official leaderboard
AMC 12 2024	143.7	#1 of 1	Lab model card

Coding

Benchmark	Score	Rank	Source
BigCodeBench-Hard	35.1%	#4 of 96	Official leaderboard
BIRD-CRITIC	32%	#2 of 9	Official leaderboard
SEAL Coding	1,100	#7 of 26	Official leaderboard
SciCode	4.60%	#5 of 18	Official leaderboard
LiveCodeBench	65.9%	#16 of 54	Lab model card
SWE-PolyBench	11.5%	#2 of 6	Official leaderboard
LiveCodeBench v5	65.9%	#16 of 47	Lab model card
METR Time Horizon	28 min	#16 of 31	Official leaderboard
Codeforces rating	2,029	#25 of 47	Lab model card
Multi-SWE-bench	5.88%	#6 of 11	Official leaderboard
Aider Polyglot	53.3%	#18 of 30	Lab model card
WeirdML	36.5%	#91 of 112	Official leaderboard
SWE-bench Verified	49.2%	#131 of 155	Lab model card
PaperBench	6.00%	#3 of 3	Official leaderboard

Agents and tools

Benchmark	Score	Rank	Source
ToolComp	65.3%	#5 of 32	Official leaderboard
Step Race	3	#19 of 59	Official leaderboard
Elimination Game	4	#30 of 56	Official leaderboard
CRMArena	35.1%	#6 of 9	Official leaderboard
lmgame-Bench	19.2	#16 of 23	Official leaderboard
LLM Chess	-216	#96 of 123	Official leaderboard
Orak	6.40	#5 of 6	Official leaderboard

Professional work

Benchmark	Score		Rank	Source
xbench-Recruitment	48.3		#6 of 8	Official leaderboard

Safety and security

Benchmark	Score	Rank	Source
BBQ (Bias Benchmark for QA)	96.6%	#1 of 25	Lab model card
HarmBench	35%	#4 of 5	Lab model card
XSTest	95.3%	#14 of 16	Lab model card
FORTRESS	74.4%	#73 of 73	Official leaderboard
Do-Not-Answer	94.8%	#2 of 2	Lab model card
Anthropic Red Team	96.2%	#1 of 1	Lab model card
DeepSeek in-house safety benchmark	25.2%	#1 of 1	Lab model card
DeepSeek jailbreak robustness evaluation	85.9%	#1 of 1	Lab model card
DeepSeek multilingual safety evaluation	74.2%	#1 of 1	Lab model card
SimpleSafetyTests	97.5%	#1 of 1	Lab model card

Other

Benchmark	Score		Rank	Source
CAIS Risk Index	66.9		#62 of 67	Official leaderboard
CAIS Text Capabilities Index	11.8		#63 of 68	Official leaderboard

Its lab's numbers put it at 136.2, 2.3 points below its ECI

The Self-Reported Index fits one score to the 19 capability results DeepSeek reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 134.4 to 138.1. How we compute it.

Its lab's numbers next to 2 leaderboards

Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.

Benchmark	Lab says	Board says	Gap
Aider Polyglot	53.3% Aider-Polyglot (Acc.)	56.9% Leaderboard, aider, diff edits	−3.6 pts
GPQA Diamond	71.5% GPQA-Diamond (Pass@1)	71.7% Epoch AI run	−0.2 pts

DeepSeek's models over time

DeepSeek's models in color, against every other model on the Epoch Capabilities Index, by release date.

Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

DeepSeek-R1 model card Jan 20, 2025

Its system card

Every result in the longer documents the lab published with it. 12 of them aren't in the launch post.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (arXiv 2501.12948) Jan 20, 2025

Capabilities