o4-mini

o4-mini ranks first on 10 of the 211 benchmarks with five or more models scored.

OpenAI Released April 16th, 2025 Compare with others

216

Benchmarks with a score

Ranked first, out of five or more models

In the top three

OpenAI Apr 2025

o4-mini

Closed weights

145.3 #94 of 256

Epoch Capabilities Index

Self-Reported Index 137.7 #198 of 404

Humanity's Last Exam 18.1% #23 /47
MultiChallenge 44.9% #33 /42
Toolathlon 14.8% #44 /50

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

Benchmark	Score	Rank	Source
IFEval	92.8%	#5 of 60	Official leaderboard
Arena-Hard v2.0	79.1%	#2 of 23	Official leaderboard
TableBench	61.7%	#1 of 10	Official leaderboard
Video SimpleQA	54%	#3 of 27	Official leaderboard
Seal-0	5.40%	#2 of 11	Official leaderboard
Fiction.LiveBench	77.8%	#12 of 57	Fiction.live leaderboard, via Epoch AI
AraGen	70.6%	#11 of 42	Official leaderboard
Wer wird Millionär? LLM Benchmark	56178.0	#25 of 77	Official leaderboard
ECLeKTic	22.6%	#12 of 35	Official leaderboard
MARS-Bench	47.1%	#6 of 15	Official leaderboard
LMArena (Elo)	1,391	#141 of 335	Official leaderboard
DiscoX	55.1	#3 of 7	Official leaderboard
LLM Confabulation Leaderboard	15.8%	#26 of 58	Official leaderboard
MMMLU	85.2%	#33 of 71	Lab model card
WritingBench	72.9	#21 of 45	Official leaderboard
EQ-Bench 3	1,279	#37 of 78	Official leaderboard
Humanity's Last Exam	18.1%	#23 of 47	Official leaderboard
HLE Calibration	57.0	#2 of 4	Official leaderboard
SnorkelSpatial	14.8%	#15 of 27	Official leaderboard
Vectara Hallucination Leaderboard (HHEM)	4.60%	#84 of 148	Official leaderboard
Global MMLU Lite	87%	#27 of 43	Official leaderboard
FACTS Search	66.2%	#20 of 31	Official leaderboard
SimpleQA	20%	#40 of 60	Lab model card
PersonQA	36%	#6 of 9	Lab model card
SimpleQA Verified	21.2%	#39 of 56	Official leaderboard
FaithJudge	25.8%	#26 of 37	Official leaderboard
MultiNRC	22.2%	#27 of 38	Official leaderboard
Visual Aesthetic Benchmark	25%	#13 of 18	Official leaderboard
FACTS Parametric	20.5%	#33 of 45	Official leaderboard
IndicGenBench	41.9	#10 of 13	Official leaderboard
MultiChallenge	44.9%	#33 of 42	Official leaderboard
Judgemark v2.1	56.1	#35 of 42	Official leaderboard
BullshitBench	10.9%	#96 of 107	Official leaderboard
HUMAINE	20	#45 of 50	Official leaderboard
BullshitBench V2	8.00%	#117 of 129	Official leaderboard
FACTS Grounding	62.1%	#31 of 34	Official leaderboard
SimpleQA Hallucination Rate	79%	#11 of 12	Lab model card
Vectara Hallucination Leaderboard (2025 dataset)	18.6%	#97 of 104	Official leaderboard

Math and science

Benchmark	Score	Rank	Source
MATH Level 5	97.8%	#3 of 94	Epoch AI run
AIME 2024	93.4%	#4 of 88	Lab model card
SnorkelGraph	75%	#2 of 24	Official leaderboard
GPQA Main	73.5%	#6 of 60	Official leaderboard
PhyX	45.8%	#1 of 10	Official leaderboard
UGMathBench	81.9%	#1 of 7	Official leaderboard
IneqMath	15.5%	#7 of 34	Official leaderboard
PolyMath	45.6%	#9 of 38	Official leaderboard
AMO-Bench	40.2%	#5 of 21	Official leaderboard
AIME 2025	92.7%	#37 of 143	Lab model card
FutureX	26.3	#2 of 7	Official leaderboard
MATH-500	94.2%	#11 of 38	Official leaderboard
USAMO 2025	19.1%	#3 of 10	Official leaderboard
SnorkelSequences	68.8%	#8 of 25	Official leaderboard
SMT 2025	88.7%	#13 of 40	Official leaderboard
SciArena	1,029	#12 of 34	Official leaderboard
HiPhO	59.5%	#13 of 36	Official leaderboard
CMT-Benchmark	18%	#4 of 11	Official leaderboard
ForecastBench	0.13	#30 of 82	Official leaderboard
FrontierMath	24.8%	#21 of 55	Official leaderboard
PHYBench	29.4%	#11 of 28	Official leaderboard
HMMT 2025	83.3%	#21 of 53	Official leaderboard
CORE-Bench	35.6%	#7 of 17	Official leaderboard
FrontierMath Tier 4	6.25%	#23 of 55	Epoch AI run
GPQA Diamond	79.6%	#98 of 229	Epoch AI run
OTIS Mock AIME 2024-2025	81.7%	#95 of 212	Epoch AI run
PutnamBench	0.30%	#6 of 12	Official leaderboard
DTBench	77.6%	#98 of 180	Official leaderboard
CMIMC 2025	83.8%	#18 of 33	Official leaderboard
Misguided Attention	42.9%	#25 of 45	Official leaderboard
SGI-Bench	30.1%	#9 of 15	Official leaderboard
LiveMathematicianBench	24.6%	#11 of 17	Official leaderboard
General365	51%	#15 of 23	Official leaderboard
ARC-AGI-2	6.11%	#64 of 98	Official leaderboard
ARC-AGI	58.7%	#63 of 95	Official leaderboard
BRUMO 2025	86.7%	#28 of 42	Official leaderboard
IMO 2025	14.3%	#4 of 6	Official leaderboard
Putnam-like	70.4%	#4 of 6	Official leaderboard
FrontierMath Tiers 1-3 v2	36.1%	#61 of 86	Official leaderboard
SimpleBench	38.7%	#77 of 106	Official leaderboard
IMO-ProofBench	11.4%	#15 of 20	Official leaderboard
FrontierMath Tier 4 v2	4.88%	#58 of 68	Official leaderboard
FrontierScience-Olympiad	61.6%	#6 of 7	Official leaderboard
FrontierScience-Research	8.33%	#6 of 7	Official leaderboard
IMProofBench	6.45%	#8 of 9	Official leaderboard
SciPredict	16.2%	#11 of 12	Official leaderboard
miniCTX-v2	24.9%	#4 of 4	Official leaderboard

Coding

Benchmark	Score	Rank	Source
LiveCodeBench	91%	#1 of 21	Official leaderboard
LiveCodeBench v5	81.4%	#1 of 21	Official leaderboard
LiveCodeBench v6	80.2%	#1 of 21	Official leaderboard
AetherCode	35.5%	#1 of 9	Official leaderboard
OIBench	36.4%	#1 of 9	Official leaderboard
OpenAI Research Engineer Interview: Multiple Choice	83%	#1 of 9	Lab model card
Codeforces rating	2,719	#7 of 47	Lab model card
AutoCodeBench	50%	#6 of 34	Official leaderboard
OpenAI Research Engineer Interview: Coding	98%	#2 of 7	Lab model card
AlgoTune	1.72×	#6 of 18	Official leaderboard
SWE-bench Multimodal	33.9%	#3 of 9	Official leaderboard
Aider Polyglot	68.9%	#11 of 30	Lab model card
WeirdML	52.6%	#44 of 112	Official leaderboard
SWE-bench Verified	64.6%	#24 of 60	Official leaderboard
LiveSQLBench	24.2%	#19 of 36	Official leaderboard
SWE-Lancer	52%	#6 of 10	Lab model card
ALE-Bench	1,271	#78 of 126	Official leaderboard
PaperBench	25%	#10 of 15	Lab model card
OpenAI PRs	39%	#8 of 12	Lab model card
IOI (Vals)	4.83%	#42 of 55	Official leaderboard
DesignArena Game Dev	993	#124 of 143	Official leaderboard
DesignArena UI Component	974	#124 of 139	Official leaderboard
DesignArena Website	978	#135 of 151	Official leaderboard
GSO-Bench	3.60%	#27 of 30	Official leaderboard
DesignArena Code	966	#133 of 143	Official leaderboard
DesignArena Data Visualization	989	#113 of 118	Official leaderboard
DesignArena 3D Design	837	#134 of 136	Official leaderboard
SWE-Lancer Diamond (SWE Manager)	23%	#4 of 4	Lab model card

Agents and tools

Benchmark	Score	Rank	Source
SnakeBench	35	#11 of 188	Official leaderboard
SnorkelWordle	91.9%	#4 of 19	Official leaderboard
lmgame-Bench	45.2	#5 of 23	Official leaderboard
BFCL V4	53.2%	#17 of 65	Official leaderboard
LLM Chess	440	#45 of 123	Official leaderboard
Galileo Agent Leaderboard	0.89	#11 of 29	Official leaderboard
Step Race	3	#26 of 59	Official leaderboard
BrowseComp-ZH	15.2%	#4 of 9	Official leaderboard
MM-BrowseComp	7.00%	#5 of 11	Official leaderboard
LisanBench	800.0	#61 of 133	Official leaderboard
MCP-Universe	26%	#14 of 30	Official leaderboard
VITA-Bench	19.5%	#10 of 18	Official leaderboard
GAIA	42.5%	#36 of 64	Official leaderboard
TextQuests	18.6	#17 of 28	Official leaderboard
Chess Puzzles	22%	#61 of 93	Epoch AI run
AgentBench FC	39.7%	#8 of 12	Official leaderboard
MCPMark	17.3%	#19 of 28	Official leaderboard
tau-bench	57.4%	#16 of 23	Lab model card
Elimination Game	3	#44 of 56	Official leaderboard
tau2-bench	56.9%	#8 of 10	Official leaderboard
Toolathlon	14.8%	#44 of 50	Official leaderboard
VitaBench 2.0	27%	#16 of 17	Official leaderboard
Mystery Game Puzzles	5.00%	#81 of 84	Epoch AI run
BrowseComp	28.3%	#79 of 81	Lab model card
DeepPlanning	13.7%	#15 of 15	Official leaderboard
CocoaBench	3.30%	#6 of 6	Official leaderboard

Professional work

Benchmark	Score	Rank	Source
MedHELM	69.7	#2 of 13	Official leaderboard
TaxEval v2	74.8%	#29 of 129	Official leaderboard
CadEval	62%	#3 of 13	Official leaderboard
SnorkelUnderwrite	78%	#7 of 27	Official leaderboard
FinQA (Snorkel OpenEnv environment)	76.6%	#8 of 22	Official leaderboard
SnorkelFinance	76.6%	#8 of 22	Official leaderboard
xbench-Recruitment	61.4	#3 of 8	Official leaderboard
Finance Reasoning (Snorkel)	45.6%	#9 of 21	Official leaderboard
Vals Finance Agent v1	35.9%	#21 of 47	Official leaderboard
FailureSensorIQ	64.8%	#11 of 24	Official leaderboard
MortgageTax	64.8%	#43 of 87	Official leaderboard
ProfBench	47.5%	#37 of 72	Official leaderboard
CorpFin v2	59%	#76 of 119	Official leaderboard
LegalBench	79.2%	#92 of 138	Official leaderboard
GDPval	27.9%	#4 of 5	Official leaderboard
MedCode	33.8%	#77 of 96	Official leaderboard
PRBench	38.1%	#41 of 48	Official leaderboard
xbench-Marketing	43.5	#6 of 7	Official leaderboard
MedScribe	69.1%	#86 of 97	Official leaderboard

Vision, audio and robotics

Benchmark	Score	Rank	Source
CharXiv (DQ)	94.3%	#2 of 67	Official leaderboard
VisualPuzzles	57%	#1 of 29	Official leaderboard
MMMU	81.6%	#5 of 136	Official leaderboard
VISTA	51.8%	#5 of 54	Official leaderboard
VHELM	76.0	#4 of 41	Official leaderboard
MM-HELIX	75.2%	#2 of 17	Official leaderboard
MicroVQA	55.6%	#3 of 19	Official leaderboard
MME-CC	35%	#2 of 12	Official leaderboard
MME-Reasoning	57.5%	#2 of 11	Official leaderboard
MORSE-500	22.2%	#2 of 11	Official leaderboard
VPCT	57.5%	#6 of 27	VPCT leaderboard, via Epoch AI
VideoReasonBench	10.7%	#4 of 14	Official leaderboard
MathVista	84.3%	#21 of 70	Lab model card
MATH-Vision (MathVision)	58%	#40 of 105	Official leaderboard
ChartMuseum	61.5%	#8 of 21	Official leaderboard
VLMsAreBiased	22.3%	#2 of 5	Official leaderboard
ZeroBench	11%	#36 of 76	Official leaderboard
CaP-X (CaP-Bench)	15.3%	#5 of 10	Official leaderboard
ViVerBench	72.7%	#4 of 8	Official leaderboard
Vision Arena	1,201	#77 of 136	Official leaderboard
VisFactor	23.2%	#16 of 26	Official leaderboard
GeoBench	64%	#32 of 48	Official leaderboard
CharXiv	72%	#53 of 75	Lab model card
ERQA	30.7%	#16 of 17	Official leaderboard

Safety and security

Benchmark	Score	Rank	Source
HELM Safety	97.3	#10 of 75	Official leaderboard
Multimodal Troubleshooting Virology	58%	#4 of 27	Lab model card
SimpleSafetyTests	100%	#12 of 75	Official leaderboard
Capture the Flag (Collegiate)	48%	#3 of 13	Lab model card
AIR-Bench 2024	78.5%	#26 of 82	Official leaderboard
Capture the Flag (High School)	80%	#3 of 9	Lab model card
Apollo Research sabotage evaluations	0.17	#1 of 3	Lab model card
Spiral-Bench v1.2	53.6	#8 of 21	Official leaderboard
Spiral-Bench v1.0	73.3	#5 of 13	Official leaderboard
Lakera B3 (Backbone Breaker Benchmark)	59.6%	#14 of 35	Official leaderboard
First-Person Fairness evaluation	0.70%	#4 of 10	Lab model card
Virology Capabilities Test (VCT)	37%	#3 of 7	Official leaderboard
Human Pathogen Capabilities Test	54.5%	#37 of 86	Official leaderboard
FORTRESS	21.5%	#34 of 73	Official leaderboard
World-Class Bio	47.5%	#35 of 74	Official leaderboard
StrongREJECT	96%	#9 of 19	Lab model card
Molecular Biology Capabilities Test	48.9%	#45 of 90	Official leaderboard
Human Sourced Jailbreaks	99%	#4 of 8	Lab model card
Capture the Flag (Professional)	41%	#11 of 21	Lab model card
Tacit Knowledge and Troubleshooting	68%	#21 of 30	Lab model card
Anthropic Red Team	98.2%	#53 of 75	Official leaderboard
Challenging Refusal Evaluation	90%	#7 of 9	Lab model card
ProtocolQA Open-Ended	27%	#22 of 27	Lab model card
Cyber Range	0%	#9 of 10	Lab model card
CyberGym	2.46%	#31 of 34	Official leaderboard
Image generation refusals	98%	#2 of 2	Lab model card

Other

Benchmark	Score	Rank	Source
HELM Capabilities	81.2	#2 of 60	Official leaderboard
CAIS Risk Index	35.9	#10 of 67	Official leaderboard
ACCoRD	63.8	#79 of 152	Official leaderboard
Conceptual Reasoning Index	41.2	#97 of 151	Official leaderboard
LMCA	26.5%	#99 of 147	Official leaderboard
SAGE (Vals)	41.1%	#57 of 81	Official leaderboard
CAIS Text Capabilities Index	15.4	#59 of 68	Official leaderboard
CAIS Vision Capabilities Index	36.8	#49 of 53	Official leaderboard

Its lab's numbers put it at 137.7, 7.6 points below its ECI

The Self-Reported Index fits one score to the 18 capability results OpenAI reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 135.6 to 139.8. How we compute it.

Its lab's numbers next to 9 leaderboards

Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.

Benchmark	Lab says	Board says	Gap
SWE-bench Verified	69% SWE-bench Verified (n=477), pass@1, o4-mini no browsing launch candidate, printed chart label (Figure 19)	64.6% Leaderboard, PatchPilot-v1.1	+4.4 pts
Aider Polyglot	68.9% Aider Polyglot whole 68.9 (diff 58.2), o4-mini-high	72% Leaderboard, high, aider, diff edits	−3.1 pts
MultiChallenge	43% Scale MultiChallenge	44.9% Leaderboard, high	−1.9 pts
GPQA Diamond	81.4% GPQA Diamond (no tools)	79.6% Epoch AI run, high	+1.8 pts
AIME 2025	92.7% AIME 2025 (no tools)	91.7% Leaderboard, high	+1.0 pts
Humanity's Last Exam	17.7% Best of HLE no tools 14.28 / with python + browsing tools 17.7	18.1% Leaderboard, high	−0.4 pts
SimpleQA	20% SimpleQA, accuracy (Table 4)	20.2% Leaderboard	−0.2 pts
CharXiv	72% CharXiv-Reasoning	72% Leaderboard, high	Same
MMMU	81.6% MMMU	81.6% Leaderboard	Same

OpenAI's models over time

OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.

Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Introducing OpenAI o3 and o4-mini (OpenAI) Apr 16, 2025

Its system card

Every result in the longer documents the lab published with it. 33 of them aren't in the launch post.

o4-mini

o4-mini

Where it ranks

Knowledge and language

Math and science

Coding

Agents and tools

Professional work

Vision, audio and robotics

Safety and security

Other

Its lab's numbers put it at 137.7, 7.6 points below its ECI

Its lab's numbers next to 9 leaderboards

OpenAI's models over time

Its launch post

Its system card

Capabilities

Hallucination

Safety

Robustness

Bias

Alignment

Bio and chem risk

Cyber risk

AI R&D risk

Autonomy