- MMMU, 0-shot, accuracy 73.4
- MMMU Pro (avg of Standard and Vision) 59.6
- MathVista 73.7
- ChartQA, relaxed_accuracy 90
- DocVQA (test), anls 94.4
- LiveCodeBench (10/01/2024-02/01/2025), pass@1
- MMLU Pro, 0-shot, macro_avg/acc 80.5
- GPQA Diamond 69.8
- MGSM, average/em 92.3
- Best of MTOB (half book) eng->kgv/kgv->eng = 54.0/46.4 / MTOB (full book) eng->kgv/kgv->eng = 50.8/46.7 54
- MMLU (5-shot) [pretrained] 85.5
- MATH (4-shot, em_maj1@1) [pretrained] 61.2
- MBPP (3-shot, pass@1) [pretrained] 77.6
- TydiQA (1-shot, average/f1) [pretrained] 31.7
Llama 4 Maverick
Llama 4 Maverick places in the top three on 3 benchmarks.
141
Benchmarks with a score
0
Ranked first, out of five or more models
3
In the top three
Meta Apr 2025 Llama 4 Maverick
Open weights Open model
132.1
#153 of 256
Epoch Capabilities Index
Self-Reported Index 131.5 #265 of 404
- Humanity's Last Exam 5.68% #39 /47
- SWE-Bench Pro 5.24% #24 /24
- MMMU-Pro 59.6% #68 /90
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MGSM | 92.3% | #4 of 57 | Lab model card | |
| IFEval | 90.8% | #8 of 60 | Official leaderboard | |
| BullshitBench | 54.5% | #22 of 107 | Official leaderboard | |
| Phare | 70.8% | #16 of 62 | Official leaderboard | |
| MTOB | 54.0 | #1 of 3 | Lab model card | |
| Vectara Hallucination Leaderboard (2025 dataset) | 8.20% | #40 of 104 | Official leaderboard | |
| HELM Arabic | 79.1 | #11 of 28 | Official leaderboard | |
| MMLU | 85.5% | #52 of 131 | Lab model card | |
| Slop Score | 49.2 | #7 of 17 | Official leaderboard | |
| MMLU-Pro | 80.5% | #78 of 167 | Lab model card | |
| TydiQA | 31.7% | #1 of 2 | Lab model card | |
| Vectara Hallucination Leaderboard (HHEM) | 4.60% | #83 of 148 | Official leaderboard | |
| HELM Arabic Enterprise | 64.8 | #13 of 23 | Official leaderboard | |
| FaithJudge | 20.6% | #22 of 37 | Official leaderboard | |
| LMArena (Elo) | 1,327 | #200 of 335 | Official leaderboard | |
| Wer wird Millionär? LLM Benchmark | 1422.0 | #46 of 77 | Official leaderboard | |
| BullshitBench V2 | 29% | #79 of 129 | Official leaderboard | |
| SEA-HELM | 61.3 | #49 of 71 | Official leaderboard | |
| Last Translation Benchmark | 1.98 | #26 of 36 | Official leaderboard | |
| Arena-Hard v2.0 | 17.2% | #17 of 23 | Official leaderboard | |
| LLM Confabulation Leaderboard | 22.6% | #44 of 58 | Official leaderboard | |
| Fiction.LiveBench | 46.2% | #46 of 57 | Fiction.live leaderboard, via Epoch AI | |
| WritingBench | 52.5 | #37 of 45 | Official leaderboard | |
| Humanity's Last Exam | 5.68% | #39 of 47 | Official leaderboard | |
| Creative Writing v3 | 51.7 | #103 of 120 | Official leaderboard | |
| StudyArena Text | 1,184 | #74 of 85 | Official leaderboard | |
| EQ-Bench 3 | 865 | #70 of 78 | Official leaderboard | |
| HUMAINE | 18 | #46 of 50 | Official leaderboard | |
| Longform Creative Writing (EQ-Bench) | 31.9 | #110 of 119 | Official leaderboard | |
| SnorkelSpatial | 12.1% | #26 of 27 | Official leaderboard | |
| Extended NYT Connections | 8.00% | #100 of 103 | Official leaderboard | |
| Hemingway-bench | 803 | #42 of 42 | Official leaderboard | |
| MultiNRC | 8.44% | #38 of 38 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GPQA Main | 65% | #19 of 60 | Official leaderboard | |
| MATH Level 5 | 73% | #32 of 94 | Epoch AI run | |
| SnorkelSequences | 62% | #13 of 25 | Official leaderboard | |
| LiveMathematicianBench | 26.4% | #9 of 17 | Official leaderboard | |
| GPQA Diamond | 67% | #123 of 229 | Epoch AI run | |
| CMT-Benchmark | 12% | #6 of 11 | Official leaderboard | |
| MATH | 61.2% | #56 of 99 | Lab model card | |
| IneqMath | 2.50% | #22 of 34 | Official leaderboard | |
| SnorkelGraph | 34% | #16 of 24 | Official leaderboard | |
| PolyMath | 26.1% | #27 of 38 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 20.6% | #154 of 212 | Epoch AI run | |
| ForecastBench | 0.15 | #60 of 82 | Official leaderboard | |
| DTBench | 61.9% | #135 of 180 | Official leaderboard | |
| MATH-500 | 85.2% | #29 of 38 | Official leaderboard | |
| Misguided Attention | 29.5% | #36 of 45 | Official leaderboard | |
| SimpleBench | 27.7% | #85 of 106 | Official leaderboard | |
| FrontierMath | 0.69% | #63 of 70 | Epoch AI run | |
| ARC-AGI-2 | 0% | #92 of 98 | Official leaderboard | |
| ARC-AGI | 4.38% | #92 of 95 | Official leaderboard | |
| SciArena | 844 | #33 of 34 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BigCodeBench | 55.5% | #3 of 69 | Official leaderboard | |
| SArena | 0.86 | #3 of 17 | Official leaderboard | |
| BigCodeBench-Hard | 28.4% | #28 of 96 | Official leaderboard | |
| MBPP | 77.6% | #28 of 71 | Lab model card | |
| Multi-SWE-bench | 6.13% | #5 of 11 | Official leaderboard | |
| Multi-SWE-bench mini | 10.3% | #3 of 5 | Official leaderboard | |
| BigO(Bench) | 32.8% | #14 of 18 | Official leaderboard | |
| Aider Polyglot | 15.6% | #39 of 46 | Official leaderboard | |
| LiveSQLBench | 13.2% | #31 of 36 | Official leaderboard | |
| SWE-bench Verified | 21% | #54 of 60 | Official leaderboard | |
| WeirdML | 24.5% | #101 of 112 | Official leaderboard | |
| DesignArena Website | 864 | #143 of 151 | Official leaderboard | |
| DesignArena 3D Design | 884 | #130 of 136 | Official leaderboard | |
| DesignArena UI Component | 890 | #134 of 139 | Official leaderboard | |
| DesignArena Code | 870 | #140 of 143 | Official leaderboard | |
| DesignArena Game Dev | 828 | #141 of 143 | Official leaderboard | |
| ALE-Bench | 552 | #126 of 126 | Official leaderboard | |
| SWE-Bench Pro | 5.24% | #24 of 24 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| TextQuests | 12.1 | #5 of 28 | Official leaderboard | |
| BFCL V4 | 37.3% | #31 of 65 | Official leaderboard | |
| SnakeBench | 23 | #105 of 188 | Official leaderboard | |
| Elimination Game | 4 | #34 of 56 | Official leaderboard | |
| LLM Chess | -100 | #75 of 123 | Official leaderboard | |
| Berkeley Function Calling Leaderboard (BFCL) | 52.9% | #43 of 67 | Official leaderboard | |
| Step Race | 1 | #47 of 59 | Official leaderboard | |
| LisanBench | 131.0 | #107 of 133 | Official leaderboard | |
| Galileo Agent Leaderboard | 0.74 | #26 of 29 | Official leaderboard | |
| Meta Agents Research Environments (ARE) / Gaia2 | 7.40% | #12 of 13 | Official leaderboard | |
| lmgame-Bench | 11.3 | #22 of 23 | Official leaderboard | |
| PACT | 1,394 | #26 of 26 | Official leaderboard | |
| Agent Poker Bench | 891 | #17 of 17 | Official leaderboard | |
| VideoGameBench | 0% | #5 of 5 | Official leaderboard | |
| VideoGameBench Lite | 0% | #5 of 5 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LEXam | 47.3% | #13 of 30 | Official leaderboard | |
| SnorkelUnderwrite | 46.3% | #19 of 27 | Official leaderboard | |
| LegalBench | 77.8% | #98 of 138 | Official leaderboard | |
| MedCode | 36.5% | #72 of 96 | Official leaderboard | |
| MortgageTax | 58.5% | #66 of 87 | Official leaderboard | |
| TaxEval v2 | 66.6% | #101 of 129 | Official leaderboard | |
| CorpFin v2 | 49.7% | #98 of 119 | Official leaderboard | |
| RadLE 2.0 | 263.0 | #10 of 12 | Official leaderboard | |
| Vals Finance Agent v1 | 3.63% | #44 of 47 | Official leaderboard | |
| ProfBench | 34.4% | #69 of 72 | Official leaderboard | |
| MedScribe | 54.2% | #95 of 97 | Official leaderboard | |
| PRBench | 24.8% | #48 of 48 | Official leaderboard | |
| Physical-Engineering's Last Exam | 0.01 | #18 of 18 | Official leaderboard | |
| HealthBench | 24.9% | #8 of 8 | Official leaderboard | |
| HealthBench Hard | 0% | #8 of 8 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ChartQA | 90% | #6 of 37 | Lab model card | |
| MMMU | 73.4% | #26 of 136 | Official leaderboard | |
| OmniAI OCR Benchmark | 82.3% | #1 of 4 | Official leaderboard | |
| DocVQA | 94.4% | #14 of 36 | Lab model card | |
| VHELM | 52.4 | #20 of 41 | Official leaderboard | |
| MeasureBench Synthetic | 12.1% | #10 of 19 | Official leaderboard | |
| MathVista | 73.7% | #38 of 70 | Lab model card | |
| MeasureBench Real-world | 12.2% | #13 of 19 | Official leaderboard | |
| VISTA | 38.3% | #37 of 54 | Official leaderboard | |
| Vision Arena | 1,145 | #102 of 136 | Official leaderboard | |
| MMMU-Pro | 59.6% | #68 of 90 | Lab model card | |
| RoboRewardBench | 1.27 | #19 of 22 | Official leaderboard | |
| GeoBench | 52% | #42 of 48 | Official leaderboard | |
| ZeroBench | 2.00% | #68 of 76 | Official leaderboard | |
| MineBench | 628 | #74 of 75 | Official leaderboard | |
| SpatialViz-Bench | 31.8% | #8 of 8 | Claim in arxiv.org, via Epoch AI | |
| Butter-Bench | 7.00% | #4 of 4 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SimpleSafetyTests | 99.3% | #29 of 75 | Official leaderboard | |
| AIR-Bench 2024 | 68.6% | #36 of 82 | Official leaderboard | |
| Spiral-Bench v1.2 | 38.2 | #11 of 21 | Official leaderboard | |
| nocot-bench (NCRI) | 70.5 | #97 of 185 | Official leaderboard | |
| HELM Safety | 90.6 | #43 of 75 | Official leaderboard | |
| BaxBench | 21.9% | #22 of 34 | Official leaderboard | |
| FORTRESS | 40.1% | #50 of 73 | Official leaderboard | |
| Spiral-Bench v1.0 | 48.1 | #9 of 13 | Official leaderboard | |
| Anthropic Red Team | 98.2% | #55 of 75 | Official leaderboard | |
| Human Pathogen Capabilities Test | 41.1% | #68 of 86 | Official leaderboard | |
| KORA | 22.9% | #34 of 42 | Official leaderboard | |
| Molecular Biology Capabilities Test | 40.2% | #74 of 90 | Official leaderboard | |
| LiveSecBench | 28.2 | #30 of 34 | Official leaderboard | |
| DistressBench | 41.3% | #42 of 42 | Official leaderboard | |
| Dictatorship Eval | 0% | #20 of 20 | Official leaderboard | |
| Anthropic political even-handedness eval | 66% | #5 of 5 | Official leaderboard | |
| AutoPatchBench | 10.6% | #3 of 3 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HELM Capabilities | 71.8 | #19 of 60 | Official leaderboard | |
| ACCoRD | 50.2 | #113 of 152 | Official leaderboard | |
| SAGE (Vals) | 35.7% | #65 of 81 | Official leaderboard | |
| Conceptual Reasoning Index | 26.8 | #122 of 151 | Official leaderboard | |
| LMCA | 15.9% | #122 of 147 | Official leaderboard | |
| TutorBench | 40.2% | #20 of 20 | Official leaderboard |
Its lab's numbers put it at 131.5, in line with its ECI
The Self-Reported Index fits one score to the 11 capability results Meta reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 128.1 to 134.9. How we compute it.
Its lab's numbers next to 4 leaderboards
Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.
Meta's models over time
Meta's models in color, against every other model on the Epoch Capabilities Index, by release date.
Source: Epoch Capabilities Index, Epoch AI (CC BY 4.0)
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it.