Gemini 2.0 Flash Feb 2025

Gemini 2.0 Flash ranks first on 3 of the 101 benchmarks with five or more models scored.

Google DeepMind Released February 5th, 2025 Compare with others

104

Benchmarks with a score

Ranked first, out of five or more models

In the top three

Google DeepMind Feb 2025

Gemini 2.0 Flash

Feb 2025 snapshot Closed weights

133.9 #146 of 256

Epoch Capabilities Index

Self-Reported Index 130.4 #271 of 404

MultiChallenge 36.4% #38 /42
ZeroBench 3.00% #57 /76
GPQA Diamond 64.1% #132 /229

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

Benchmark	Score	Rank	Source
Vectara Hallucination Leaderboard (HHEM)	0.70%	#1 of 148	Official leaderboard
SimpleVQA	52.8%	#1 of 13	Official leaderboard
FaithJudge	10.2%	#5 of 37	Official leaderboard
AidanBench	1895.0	#2 of 13	Official leaderboard
BIG-Bench Extra Hard	9.80%	#2 of 11	Official leaderboard
ECLeKTic	24.6%	#9 of 35	Official leaderboard
BBEH micro average	23.9%	#3 of 11	Official leaderboard
BuzzBench	54.4	#6 of 20	Official leaderboard
IFEval	84.1%	#23 of 60	Official leaderboard
SimpleQA	29.9%	#24 of 60	Lab model card
MultiLoKo	30.6%	#3 of 7	Official leaderboard
SuperGPQA	47.7%	#26 of 59	Official leaderboard
MRCR (Michelangelo, original)	70.5%	#4 of 9	Lab model card
FACTS Grounding	84.6%	#7 of 15	Lab model card
Answer Factuality (Perplexity)	78.5%	#2 of 4	Official leaderboard
LMArena (Elo)	1,360	#168 of 335	Official leaderboard
Fiction.LiveBench	61.1%	#32 of 57	Fiction.live leaderboard, via Epoch AI
Global MMLU Lite	83.4%	#9 of 16	Lab model card
MMLU-Pro	77.6%	#94 of 167	Lab model card
HUMAINE	26	#29 of 50	Official leaderboard
Phare	64.2%	#37 of 62	Official leaderboard
WritingBench	66.8	#29 of 45	Official leaderboard
MARS-Bench	35.6%	#11 of 15	Official leaderboard
Creative Writing v3	57.0	#91 of 120	Official leaderboard
BullshitBench V2	15%	#103 of 129	Official leaderboard
EQ-Bench 3	958	#65 of 78	Official leaderboard
BullshitBench	12.7%	#91 of 107	Official leaderboard
LLM Confabulation Leaderboard	26.9%	#50 of 58	Official leaderboard
MultiChallenge	36.4%	#38 of 42	Official leaderboard

Math and science

Benchmark	Score	Rank	Source
MATH	90.9%	#9 of 99	Lab model card
HiddenMath	63.5%	#3 of 11	Lab model card
MATH Level 5	82.2%	#26 of 94	Epoch AI run
GPQA Main	55.6%	#29 of 60	Official leaderboard
GPQA Diamond	64.1%	#132 of 229	Epoch AI run
MATH-500	89%	#24 of 38	Official leaderboard
OTIS Mock AIME 2024-2025	31.1%	#145 of 212	Epoch AI run
DTBench	63.2%	#129 of 180	Official leaderboard
FrontierMath	1.72%	#57 of 70	Epoch AI run
ARC-AGI-2	1.30%	#79 of 97	Collected by Epoch AI, source not named
Misguided Attention	28.2%	#37 of 45	Official leaderboard
CORE-Bench	11.1%	#14 of 17	Official leaderboard
AIME 2025	27.5%	#49 of 53	Official leaderboard
HMMT 2025	13.3%	#49 of 53	Official leaderboard

Coding

Benchmark	Score	Rank	Source
BigCodeBench-Hard	28.7%	#25 of 96	Official leaderboard
Bird-SQL	58.7%	#5 of 13	Lab model card
LiveCodeBench v5	34.5%	#40 of 47	Lab model card
StructEval	62.5%	#24 of 28	Official leaderboard
SWE-bench Verified	13.5%	#56 of 60	Official leaderboard

Agents and tools

Benchmark	Score	Rank	Source
SnakeBench	33	#16 of 188	Official leaderboard
Galileo Agent Leaderboard	0.94	#3 of 29	Official leaderboard
Berkeley Function Calling Leaderboard (BFCL)	61.2%	#22 of 67	Official leaderboard
Factorio Learning Environment	115782.0	#2 of 6	Official leaderboard
Research-Eval	54.2%	#1 of 3	Official leaderboard
LLM-WikiRace	6.00%	#9 of 21	Official leaderboard
ToolComp	55.6%	#18 of 32	Official leaderboard
VideoGameBench	0%	#3 of 5	Official leaderboard
LisanBench	197.7	#89 of 133	Official leaderboard
VideoGameBench Lite	0%	#4 of 5	Official leaderboard
LLM Chess	-311	#102 of 123	Official leaderboard
BrowseComp-ZH	6.90%	#8 of 9	Official leaderboard
GAIA	6.31%	#59 of 64	Official leaderboard
Elimination Game	2	#54 of 56	Official leaderboard

Professional work

Benchmark	Score	Rank	Source
MedQA	73%	#21 of 71	Official leaderboard
MedXpertQA	28%	#2 of 4	Official leaderboard
LegalBench	78.4%	#95 of 138	Official leaderboard
MortgageTax	60.4%	#62 of 87	Official leaderboard
TaxEval v2	67.7%	#96 of 129	Official leaderboard
CadEval	30%	#10 of 13	Official leaderboard
MedHELM	36.0	#10 of 13	Official leaderboard
Vals Finance Agent v1	13.2%	#37 of 47	Official leaderboard
FailureSensorIQ	55.8%	#19 of 24	Official leaderboard
CorpFin v2	33.7%	#118 of 119	Official leaderboard

Vision, audio and robotics

Benchmark	Score	Rank	Source
KRETA	85.4%	#1 of 17	Official leaderboard
OCR-Reasoning	39.3%	#2 of 15	Official leaderboard
MathVista	73.1%	#6 of 43	Official leaderboard
CSVQA	44.1%	#2 of 12	Official leaderboard
AHELM	63.3	#2 of 10	Official leaderboard
MMMU	71.7%	#29 of 136	Official leaderboard
VHELM	68.1	#10 of 41	Official leaderboard
EmoBench-M	62.3%	#4 of 15	Official leaderboard
ERQA	40.2%	#5 of 17	Official leaderboard
VisualPuzzles	44.9%	#9 of 29	Official leaderboard
GeoBench	77%	#17 of 48	Official leaderboard
CoVoST2	39.0	#5 of 14	Lab model card
MathCanvas-Bench	21.2%	#6 of 16	Official leaderboard
MMAU-Pro	55.7%	#5 of 10	Official leaderboard
Point-Bench	47%	#9 of 17	Official leaderboard
MATH-Vision (MathVision)	41.3%	#59 of 105	Official leaderboard
VISTA	39.9%	#34 of 54	Official leaderboard
Math-VR	20.6%	#17 of 26	Official leaderboard
Vision Arena	1,172	#92 of 136	Official leaderboard
ZeroBench	3.00%	#57 of 76	Official leaderboard
EgoSchema	71.1%	#9 of 12	Lab model card
ViewSpatial-Bench	32.6%	#6 of 7	Official leaderboard

Safety and security

Benchmark	Score	Rank	Source
Anthropic Red Team	99.4%	#23 of 75	Official leaderboard
Video-SafetyBench	2.00	#6 of 15	Official leaderboard
AIR-Bench 2024	66.2%	#41 of 82	Official leaderboard
HELM Safety	91.0	#41 of 75	Official leaderboard
SimpleSafetyTests	98.5%	#45 of 75	Official leaderboard
BaxBench	23.3%	#21 of 34	Official leaderboard
AgentDojo	20.8%	#13 of 20	Official leaderboard
AIRTBench	16.9%	#6 of 9	Official leaderboard

Other

Benchmark	Score		Rank	Source
HELM Lite	81.3		#12 of 71	Official leaderboard
HELM Capabilities	67.9		#25 of 60	Official leaderboard

Its lab's numbers put it at 130.4, 3.6 points below its ECI

The Self-Reported Index fits one score to the 12 capability results Google DeepMind reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 127.6 to 133.1. How we compute it.

Its lab's numbers next to 4 leaderboards

Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.

Benchmark	Lab says	Board says	Gap
GPQA Diamond	60.1% GPQA (diamond)	64.1% Epoch AI run	−4.0 pts
FACTS Grounding	84.6% FACTS Grounding	85.6% Leaderboard	−1.0 pts
MMLU-Pro	77.6% MMLU-Pro	77.6% Leaderboard	Same
MMMU	71.7% MMMU	71.7% Leaderboard	Same

Google DeepMind's models over time

Google DeepMind's models in color, against every other model on the Epoch Capabilities Index, by release date.

Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Gemini 2.0 is now available to everyone (Google blog) Feb 5, 2025

MMLU-Pro 77.6
LiveCodeBench (v5) (UI 10/01/2024-02/01/2025) 34.5
Bird-SQL (Dev) 58.7
GPQA (diamond) 60.1
SimpleQA 29.9
FACTS Grounding 84.6
Global MMLU (Lite) 83.4
MATH 90.9
HiddenMath 63.5
MRCR (1M) 70.5
MMMU 71.7
CoVoST2 (21 lang), BLEU 39
EgoSchema (test) 71.1

Its system card

Every result in the longer documents the lab published with it. 5 of them aren't in the launch post.

Gemini 2.0 Flash Model Card Apr 15, 2025