- MMMU, 0-shot, accuracy 69.4
- MMMU Pro (avg of Standard and Vision) 52.2
- MathVista 70.7
- ChartQA, relaxed_accuracy 88.8
- DocVQA (test), anls 94.4
- LiveCodeBench (10/01/2024-02/01/2025), pass@1 32.8
- MMLU Pro, 0-shot, macro_avg/acc 74.3
- GPQA Diamond 57.2
- MGSM, average/em 90.6
- Best of MTOB (half book) eng->kgv/kgv->eng = 42.2/36.6 / MTOB (full book) = 39.7/36.3 42.2
- MMLU (5-shot) [pretrained] 79.6
- MATH (4-shot, em_maj1@1) [pretrained] 50.3
- MBPP (3-shot, pass@1) [pretrained] 67.8
- TydiQA (1-shot, average/f1) [pretrained] 31.5
Llama 4 Scout
Llama 4 Scout ranks first on TextQuests.
94
Benchmarks with a score
1
Ranked first, out of five or more models
2
In the top three
Meta Apr 2025 Llama 4 Scout
Open weights
127.7
#163 of 256
Epoch Capabilities Index
Self-Reported Index 123.6 #326 of 404
- MMMU-Pro 52.2% #76 /90
- ZeroBench 3.00% #58 /76
- GPQA Diamond 51.8% #152 /229
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MGSM | 90.6% | #10 of 57 | Lab model card | |
| Vectara Hallucination Leaderboard (2025 dataset) | 7.70% | #38 of 104 | Official leaderboard | |
| BullshitBench | 36.4% | #40 of 107 | Official leaderboard | |
| SnorkelSpatial | 15.4% | #14 of 27 | Official leaderboard | |
| MMLU | 79.6% | #78 of 131 | Lab model card | |
| IFEval | 81.8% | #36 of 60 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 4.70% | #89 of 148 | Official leaderboard | |
| LMArena (Elo) | 1,322 | #209 of 335 | Official leaderboard | |
| Phare | 63.8% | #39 of 62 | Official leaderboard | |
| MMLU-Pro | 74.3% | #110 of 167 | Lab model card | |
| SEA-HELM | 63.4 | #47 of 71 | Official leaderboard | |
| BullshitBench V2 | 20% | #92 of 129 | Official leaderboard | |
| HELM Arabic | 75.6 | #21 of 28 | Official leaderboard | |
| HELM Arabic Enterprise | 62.1 | #18 of 23 | Official leaderboard | |
| Seal-0 | 0% | #9 of 11 | Official leaderboard | |
| WritingBench | 52.0 | #38 of 45 | Official leaderboard | |
| Fiction.LiveBench | 36% | #53 of 57 | Fiction.live leaderboard, via Epoch AI | |
| Creative Writing v3 | 46.6 | #113 of 120 | Official leaderboard | |
| EQ-Bench 3 | 670 | #74 of 78 | Official leaderboard | |
| Longform Creative Writing (EQ-Bench) | 31.1 | #113 of 119 | Official leaderboard | |
| StudyArena Text | 1,159 | #81 of 85 | Official leaderboard | |
| MTOB | 42.2 | #3 of 3 | Lab model card | |
| TydiQA | 31.5% | #2 of 2 | Lab model card |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MATH Level 5 | 62.3% | #42 of 94 | Epoch AI run | |
| GPQA Main | 50.7% | #35 of 60 | Official leaderboard | |
| SnorkelSequences | 48.4% | #15 of 25 | Official leaderboard | |
| Misguided Attention | 35.9% | #29 of 45 | Official leaderboard | |
| GPQA Diamond | 51.8% | #152 of 229 | Epoch AI run | |
| MATH | 50.3% | #69 of 99 | Lab model card | |
| SnorkelGraph | 26% | #19 of 24 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 7.78% | #168 of 212 | Epoch AI run | |
| HiPhO | 29.9% | #29 of 36 | Official leaderboard | |
| PolyMath | 20.9% | #31 of 38 | Official leaderboard | |
| DTBench | 57.9% | #148 of 180 | Official leaderboard | |
| MATH-500 | 79.2% | #32 of 38 | Official leaderboard | |
| IneqMath | 1.50% | #30 of 34 | Official leaderboard | |
| ForecastBench | 0.17 | #74 of 82 | Official leaderboard | |
| SGI-Bench | 21.1% | #14 of 15 | Official leaderboard | |
| ARC-AGI-2 | 0% | #93 of 98 | Official leaderboard | |
| ARC-AGI | 0.50% | #94 of 95 | Official leaderboard | |
| FrontierMath | 0% | #70 of 70 | Epoch AI run | |
| SciArena | 830 | #34 of 34 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SArena | 0.84 | #5 of 17 | Official leaderboard | |
| MBPP | 67.8% | #48 of 71 | Lab model card | |
| BigCodeBench-Hard | 16.9% | #68 of 96 | Official leaderboard | |
| LiveSQLBench | 18.9% | #28 of 36 | Official leaderboard | |
| LiveCodeBench v5 | 32.8% | #43 of 47 | Lab model card | |
| BigO(Bench) | 23% | #17 of 18 | Official leaderboard | |
| SWE-bench Verified | 9.06% | #57 of 60 | Official leaderboard | |
| DesignArena Website | 743 | #151 of 151 | Official leaderboard | |
| DesignArena Code | 780 | #143 of 143 | Official leaderboard | |
| DesignArena Game Dev | 763 | #143 of 143 | Official leaderboard | |
| DesignArena UI Component | 759 | #139 of 139 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| TextQuests | 7.00 | #1 of 28 | Official leaderboard | |
| SnakeBench | 21 | #123 of 188 | Official leaderboard | |
| BFCL V4 | 28.1% | #43 of 65 | Official leaderboard | |
| LisanBench | 126.3 | #109 of 133 | Official leaderboard | |
| Berkeley Function Calling Leaderboard (BFCL) | 45.4% | #57 of 67 | Official leaderboard | |
| LLM Chess | -402 | #107 of 123 | Official leaderboard | |
| PinchBench 2.0 | 3.19% | #57 of 57 | Official leaderboard | |
| Galileo Agent Leaderboard | 0.63 | #29 of 29 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MultiFinBen | 20.9 | #2 of 9 | Official leaderboard | |
| BizFinBench | 61.2% | #8 of 13 | Official leaderboard | |
| SnorkelUnderwrite | 44.3% | #21 of 27 | Official leaderboard | |
| MortgageTax | 57.8% | #68 of 87 | Official leaderboard | |
| LegalBench | 72% | #109 of 138 | Official leaderboard | |
| CorpFin v2 | 46.8% | #100 of 119 | Official leaderboard | |
| TaxEval v2 | 55.2% | #123 of 129 | Official leaderboard | |
| MedCode | 23.3% | #93 of 96 | Official leaderboard | |
| ProfBench | 31.2% | #71 of 72 | Official leaderboard | |
| MedScribe | 50.6% | #96 of 97 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMMU | 69.4% | #35 of 136 | Official leaderboard | |
| ChartQA | 88.8% | #11 of 37 | Lab model card | |
| SpatialViz-Bench | 34.2% | #4 of 8 | Claim in arxiv.org, via Epoch AI | |
| MathVista | 70.7% | #46 of 70 | Lab model card | |
| VHELM | 38.0 | #30 of 41 | Official leaderboard | |
| ZeroBench | 3.00% | #58 of 76 | Official leaderboard | |
| Vision Arena | 1,128 | #107 of 136 | Official leaderboard | |
| MeasureBench Real-world | 10.9% | #15 of 19 | Official leaderboard | |
| MeasureBench Synthetic | 9.10% | #15 of 19 | Official leaderboard | |
| MMMU-Pro | 52.2% | #76 of 90 | Lab model card | |
| RoboRewardBench | 1.49 | #21 of 22 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SimpleSafetyTests | 97% | #56 of 75 | Official leaderboard | |
| HELM Safety | 87.3 | #57 of 75 | Official leaderboard | |
| AIRTBench | 1.00% | #7 of 9 | Official leaderboard | |
| AIR-Bench 2024 | 52.3% | #64 of 82 | Official leaderboard | |
| nocot-bench (NCRI) | 57.7 | #146 of 185 | Official leaderboard | |
| Anthropic Red Team | 96.5% | #64 of 75 | Official leaderboard | |
| KORA | 17.7% | #37 of 42 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HELM Capabilities | 64.4 | #33 of 60 | Official leaderboard | |
| ACCoRD | 49.7 | #114 of 152 | Official leaderboard | |
| SAGE (Vals) | 34.8% | #69 of 81 | Official leaderboard | |
| Conceptual Reasoning Index | 23.1 | #132 of 151 | Official leaderboard | |
| LMCA | 12% | #131 of 147 | Official leaderboard |
Its lab's numbers put it at 123.6, 4.0 points below its ECI
The Self-Reported Index fits one score to the 11 capability results Meta reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 120.2 to 127.1. How we compute it.
Its lab's numbers next to 3 leaderboards
Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.
Meta's models over time
Meta's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 1 of them isn't in the launch post.