Gemini 1.0 Pro
Gemini 1.0 Pro ranks first on 2 of the 52 benchmarks with five or more models scored.
55
Benchmarks with a score
2
Ranked first, out of five or more models
6
In the top three
Google DeepMind Dec 2023 Gemini 1.0 Pro
Closed weights
No Epoch Capabilities Index score yet
- GPQA Diamond 34% #199 /229
- MM-Vet 58.6% #59 /164
- ChemBench 45.3% #21 /24
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GAOKAO-Bench | 57.9% | #2 of 7 | Official leaderboard | |
| CriticEval | 5.57 | #5 of 12 | Official leaderboard | |
| EQ-Bench v2 | 75.1 | #37 of 85 | Official leaderboard | |
| AlpacaEval 2.0 | 24.4% | #22 of 47 | Official leaderboard | |
| MMBench-CN v1.1 | 69.2% | #104 of 178 | Official leaderboard | |
| ThaiExam | 47.7% | #15 of 24 | Official leaderboard | |
| TurkishMMLU | 63.2% | #5 of 8 | Official leaderboard | |
| MixEval | 78.9% | #23 of 34 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 7.70% | #116 of 148 | Official leaderboard | |
| LMArena (Elo) | 1,237 | #268 of 335 | Official leaderboard | |
| Arena-Hard-Auto | 17.8% | #32 of 38 | Official leaderboard | |
| AlpacaEval | 79.7% | #16 of 19 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GPQA Diamond | 34% | #199 of 229 | Epoch AI run | |
| ChemBench | 45.3% | #21 of 24 | Official leaderboard | |
| MATH Level 5 | 11.2% | #87 of 94 | Epoch AI run | |
| GSM1k | 79.8% | #14 of 15 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 1.11% | #203 of 212 | Epoch AI run | |
| DTBench | 45.9% | #176 of 180 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MBPP | 75.4% | #18 of 35 | Official leaderboard | |
| MBPP+ | 61.4% | #20 of 35 | Official leaderboard | |
| HumanEval | 63.4% | #29 of 46 | Official leaderboard | |
| HumanEval+ | 55.5% | #30 of 46 | Official leaderboard | |
| CodeEditorBench | 73.7% | #5 of 7 | Official leaderboard | |
| SEAL Coding | 685 | #25 of 26 | Official leaderboard | |
| MLAgentBench | 18.3% | #1 of 1 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SnakeBench | 35 | #10 of 188 | Official leaderboard | |
| VisualWebBench | 48% | #1 of 7 | Official leaderboard | |
| VisualWebArena | 6.04% | #3 of 7 | Official leaderboard | |
| GroundUI | 1.80% | #8 of 11 | Official leaderboard | |
| VisualAgentBench | 6.30% | #4 of 5 | Official leaderboard | |
| WebArena | 7.12% | #4 of 4 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LegalBench | 70.3% | #115 of 138 | Official leaderboard | |
| MedQA | 48.3% | #63 of 71 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BLINK | 45.7% | #2 of 11 | Official leaderboard | |
| MM-Vet | 58.6% | #59 of 164 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 52.5% | #60 of 152 | Official leaderboard | |
| MathVerse | 23.5% | #4 of 9 | Official leaderboard | |
| MMBench | 73.6% | #91 of 155 | Official leaderboard | |
| MMBench-EN v1.1 | 70.2% | #118 of 178 | Official leaderboard | |
| CharXiv (DQ) | 54.4% | #46 of 67 | Official leaderboard | |
| MMMU | 47.9% | #95 of 136 | Official leaderboard | |
| MME | 1496.6 | #12 of 17 | Official leaderboard | |
| MathVista | 45.2% | #32 of 43 | Official leaderboard | |
| CharXiv | 22.8% | #63 of 77 | Official leaderboard | |
| MMStar | 38.6% | #148 of 173 | Official leaderboard | |
| MATH-Vision (MathVision) | 17.7% | #93 of 105 | Official leaderboard | |
| V* | 48.2% | #2 of 2 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OR-Bench | 9.70% | #2 of 28 | Official leaderboard | |
| DecodingTrust | 80.6 | #1 of 5 | Official leaderboard | |
| AgentHarm | 7.40% | #4 of 8 | Official leaderboard | |
| SORRY-Bench | 32.7% | #14 of 23 | Official leaderboard | |
| AIR-Bench 2024 | 58.2% | #55 of 82 | Official leaderboard | |
| SALAD-Bench | 88.3% | #10 of 13 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 56.1 | #104 of 173 | Official leaderboard | |
| HELM Lite | 42.2 | #48 of 71 | Official leaderboard |