Gemma 1.1 7B IT
Gemma 1.1 7B IT places in the top three on CyberMetric.
16
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
Google DeepMind Apr 2024 Gemma 1.1 7B IT
Open weights Small
No Epoch Capabilities Index score yet
- Open Arabic LLM Leaderboard v1 46.5% #20 /55
- ChemBench 19.2% #24 /24
- SGP-Bench 39.3% #19 /22
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Open Arabic LLM Leaderboard v1 | 46.5% | #20 of 55 | Official leaderboard | |
| Ko-IFEval (Upstage) | 39.3% | #8 of 21 | Official leaderboard | |
| MuSR | 42.3% | #40 of 98 | Official leaderboard | |
| EQ-Bench v2 | 59.2 | #68 of 85 | Official leaderboard | |
| LMArena (Elo) | 1,183 | #293 of 335 | Official leaderboard | |
| MixEval | 69.6% | #30 of 34 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 17% | #143 of 148 | Official leaderboard | |
| Arena-Hard-Auto | 12.1% | #37 of 38 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ChemBench | 19.2% | #24 of 24 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval | 42.7% | #37 of 46 | Official leaderboard | |
| HumanEval+ | 35.4% | #38 of 46 | Official leaderboard | |
| MBPP | 57.1% | #29 of 35 | Official leaderboard | |
| MBPP+ | 45% | #29 of 35 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SGP-Bench | 39.3% | #19 of 22 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| CyberMetric | 73.3% | #2 of 10 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Open LLM Leaderboard v2 | 17.7 | #62 of 99 | Official leaderboard |