Llama 3.2 11B
Llama 3.2 11B ranks first on xMMMU.
39
Benchmarks with a score
1
Ranked first, out of five or more models
1
In the top three
Meta Sep 2024 Llama 3.2 11B
Open weights
No Epoch Capabilities Index score yet
- MicroVQA 30.3% #19 /19
- VisualPuzzles 29.4% #24 /29
- HumanEval-V 4.00% #21 /24
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| EQ-Bench v2 | 67.9 | #58 of 85 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 5.50% | #101 of 148 | Official leaderboard | |
| MMBench-CN v1.1 | 64.2% | #129 of 178 | Official leaderboard | |
| AraGen | 24.4% | #36 of 42 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| U-MATH | 20.4% | #30 of 30 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval-V | 4.00% | #21 of 24 | Official leaderboard | |
| SArena | 0.76 | #16 of 17 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BALROG | 16.8% | #25 of 33 | Official leaderboard | |
| SnakeBench | 18 | #151 of 188 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MedQA | 27% | #69 of 71 | Official leaderboard | |
| MortgageTax | 23.3% | #86 of 87 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| xMMMU | 41.4% | #1 of 5 | Official leaderboard | |
| SEED-Bench-2-Plus | 65.8% | #51 of 136 | Official leaderboard | |
| MM-Vet | 57.6% | #66 of 164 | Official leaderboard | |
| CoreCognition | 55.4% | #63 of 119 | Official leaderboard | |
| CharXiv | 31.2% | #43 of 77 | Official leaderboard | |
| MMStar | 49.8% | #105 of 173 | Official leaderboard | |
| MMMU | 50.7% | #83 of 136 | Official leaderboard | |
| VL-RewardBench | 42.8% | #5 of 8 | Official leaderboard | |
| A-Bench | 69.2% | #77 of 116 | Official leaderboard | |
| EmbodiedBench EB-Habitat | 25% | #12 of 17 | Official leaderboard | |
| MMVU | 38.9% | #20 of 28 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 30.2% | #111 of 152 | Official leaderboard | |
| MMBench-EN v1.1 | 67.5% | #134 of 178 | Official leaderboard | |
| EmbodiedBench EB-ALFRED | 13.7% | #13 of 17 | Official leaderboard | |
| EmbodiedBench EB-Navigation | 21.4% | #13 of 17 | Official leaderboard | |
| MMMU-Pro | 28.4% | #44 of 57 | Official leaderboard | |
| MMBench | 68% | #121 of 155 | Official leaderboard | |
| Do You See Me | 33.4% | #4 of 5 | Official leaderboard | |
| VisualPuzzles | 29.4% | #24 of 29 | Official leaderboard | |
| MEGA-Bench | 18% | #19 of 22 | Official leaderboard | |
| Video-MMMU | 30% | #22 of 24 | Official leaderboard | |
| EmbodiedBench EB-Manipulation | 0.90% | #16 of 17 | Official leaderboard | |
| Vision Arena | 993 | #131 of 136 | Official leaderboard | |
| VISTA | 20.5% | #53 of 54 | Official leaderboard | |
| MicroVQA | 30.3% | #19 of 19 | Official leaderboard | |
| LongDocURL | 9.20% | #5 of 5 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 57.7 | #95 of 173 | Official leaderboard | |
| HELM Lite | 32.5 | #57 of 71 | Official leaderboard |