Gemini 2.0 Flash places in the top three on 3 benchmarks.
Google DeepMind Released December 6th, 2024 Family Gemini 2.0 The lab's model card Compare with others Google DeepMind Dec 2024 Dec 2024 snapshot Closed weights Mid-size
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Vectara Hallucination Leaderboard (HHEM) | 1.30% | #9 of 148 | Official leaderboard | |
| SEAL Multilingual | 1,119 | #2 of 19 | Official leaderboard | |
| SEAL Precise Instruction Following | 86.6 | #4 of 18 | Official leaderboard | |
| LongBench v2 | 51.1% | #12 of 33 | Official leaderboard | |
| MMBench-CN v1.1 | 71.6% | #92 of 178 | Official leaderboard | |
| FACTS Grounding | 83.6% | #8 of 15 | Lab model card | |
| MRCR (Michelangelo, original) | 69.2% | #5 of 9 | Lab model card | |
| MMLU-Pro | 76.4% | #103 of 167 | Lab model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MATH | 89.7% | #10 of 99 | Lab model card | |
| HiddenMath | 63% | #4 of 11 | Lab model card | |
| GPQA Diamond | 62.1% | #214 of 302 | Lab model card | |
| ARC-AGI-2 | 1.30% | #79 of 98 | Official leaderboard | |
| SimpleBench | 18.9% | #100 of 106 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BigCodeBench | 52.9% | #8 of 69 | Official leaderboard | |
| SEAL Coding | 1,111 | #5 of 26 | Official leaderboard | |
| Natural2Code | 92.9% | #2 of 9 | Lab model card | |
| BigCodeBench-Hard | 28.4% | #26 of 96 | Official leaderboard | |
| Aider-Edit | 69.9% | #13 of 34 | Official leaderboard | |
| Bird-SQL | 56.9% | #7 of 13 | Lab model card | |
| SWE-bench Verified | 52.2% | #35 of 60 | Official leaderboard | |
| Aider Polyglot | 22.2% | #34 of 46 | Official leaderboard | |
| LiveCodeBench | 35.1% | #40 of 54 | Lab model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MedXpertQA-MM | 37.2% | #3 of 9 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMStar | 69.4% | #14 of 173 | Official leaderboard | |
| MMVU | 65.9% | #4 of 28 | Official leaderboard | |
| TOMATO | 33.7% | #4 of 22 | Official leaderboard | |
| CoVoST2 | 39.2 | #4 of 14 | Lab model card | |
| MMMU | 70.7% | #59 of 103 | Lab model card | |
| VISTA | 40% | #33 of 54 | Official leaderboard | |
| MMBench-EN v1.1 | 70.4% | #116 of 178 | Official leaderboard | |
| EgoSchema | 71.5% | #8 of 12 | Lab model card | |
| Vibe-Eval | 56.3% | #9 of 9 | Lab model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AgentDojo | 17% | #12 of 20 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 72.6 | #21 of 173 | Official leaderboard |
The Self-Reported Index fits one score to the 12 capability results Google DeepMind reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 127.1 to 132.2. How we compute it.
The benchmarks the lab chose to headline, with the scores it gave.