- MMLU (CoT), 0-shot, macro_avg/acc 86
- MMLU Pro (CoT), 5-shot, macro_avg/acc 68.9
- IFEval 92.1
- GPQA Diamond (CoT), 0-shot 50.5
- HumanEval, 0-shot, pass@1 88.4
- MBPP EvalPlus (base), 0-shot, pass@1 87.6
- MATH (CoT), 0-shot, sympy_intersection_score 77
- BFCL v2, overall_ast_summary/macro_avg/valid
- MGSM, 0-shot, em 91.1
Llama 3.3 70B Instruct
Llama 3.3 70B Instruct ranks first on SwitzerlandQA.
83
Benchmarks with a score
1
Ranked first, out of five or more models
9
In the top three
Meta Dec 2024 Llama 3.3 70B Instruct
Open weights Open model
127.3
#297 of 404
Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.
- Meta Agents Research Environments (ARE) / Gaia2 4.40% #13 /13
- LiveSQLBench 12.2% #33 /36
- Encyclo-K 20.1% #18 /23
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Vectara Hallucination Leaderboard (2025 dataset) | 4.10% | #4 of 104 | Official leaderboard | |
| ArabicMMLU | 69.8% | #3 of 39 | Official leaderboard | |
| ArbMMLU-HT | 67.8% | #3 of 39 | Official leaderboard | |
| QIMMA | 67.1% | #2 of 25 | Official leaderboard | |
| Open Arabic LLM Leaderboard v2 | 69.3% | #5 of 39 | Official leaderboard | |
| MGSM | 91.1% | #8 of 57 | Lab model card | |
| SwitzerlandQA | 74.3% | #1 of 7 | Official leaderboard | |
| IFEval | 92.1% | #20 of 135 | Lab model card | |
| MMLU-CF | 68.8% | #7 of 37 | Official leaderboard | |
| Stick to Your Role! | 0.81 | #3 of 15 | Official leaderboard | |
| MuSR | 44.6% | #20 of 98 | Official leaderboard | |
| MadinahQA | 53.1% | #10 of 39 | Official leaderboard | |
| BOUQuET | 30.6 | #5 of 18 | Official leaderboard | |
| MMLU | 86% | #50 of 131 | Lab model card | |
| LLM-AggreFact | 74.5% | #4 of 10 | Official leaderboard | |
| AraGen | 52.1% | #17 of 42 | Official leaderboard | |
| FaithJudge | 16.4% | #15 of 37 | Official leaderboard | |
| AlGhafa | 63.7% | #17 of 39 | Official leaderboard | |
| ALRAGE | 75.8% | #17 of 39 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 4.00% | #65 of 148 | Official leaderboard | |
| Alyah | 69.7% | #5 of 10 | Official leaderboard | |
| AbstentionBench | 71.4% | #3 of 5 | Official leaderboard | |
| LMArena (Elo) | 1,318 | #211 of 335 | Official leaderboard | |
| BuzzBench | 45.7 | #13 of 20 | Official leaderboard | |
| SuperGPQA | 37.7% | #39 of 59 | Official leaderboard | |
| Wer wird Millionär? LLM Benchmark | 353.0 | #52 of 77 | Official leaderboard | |
| HELM Arabic | 76.4 | #20 of 28 | Official leaderboard | |
| SEA-HELM | 60.4 | #51 of 71 | Official leaderboard | |
| Phare | 62% | #47 of 62 | Official leaderboard | |
| MMLU-Pro | 68.9% | #127 of 167 | Lab model card | |
| Encyclo-K | 20.1% | #18 of 23 | Official leaderboard | |
| HUMAINE | 21 | #41 of 50 | Official leaderboard | |
| HELM Arabic Enterprise | 61.5 | #20 of 23 | Official leaderboard | |
| WritingBench | 48.0 | #41 of 45 | Official leaderboard | |
| AidanBench | 779.0 | #12 of 13 | Official leaderboard | |
| STALE | 2.60% | #10 of 10 | Official leaderboard | |
| Answer Factuality (Perplexity) | 65.7% | #4 of 4 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MATH | 77% | #33 of 99 | Lab model card | |
| U-MATH | 44.7% | #18 of 30 | Official leaderboard | |
| Misguided Attention | 35.3% | #30 of 45 | Official leaderboard | |
| ForecastBench | 0.15 | #61 of 82 | Official leaderboard | |
| GPQA Diamond | 50.5% | #244 of 302 | Lab model card | |
| LiveMathematicianBench | 22.7% | #14 of 17 | Official leaderboard | |
| GSM8K Platinum Initial 268 | 97.4% | #17 of 19 | Official leaderboard | |
| SimpleBench | 19.9% | #99 of 106 | Official leaderboard | |
| PolyMath | 11.5% | #38 of 38 | Official leaderboard | |
| LiveMathBench | 27.5% | #5 of 5 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MBPP | 87.6% | #6 of 71 | Lab model card | |
| BigCodeBench | 52.2% | #12 of 69 | Official leaderboard | |
| HumanEval | 88.4% | #21 of 101 | Lab model card | |
| BigCodeBench-Hard | 28.4% | #27 of 96 | Official leaderboard | |
| Aider-Edit | 59.4% | #23 of 34 | Official leaderboard | |
| LiveSQLBench | 12.2% | #33 of 36 | Official leaderboard | |
| WeirdML | 14.4% | #106 of 112 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OdysseyArena Energy Dispatch | 0% | #3 of 8 | Official leaderboard | |
| PlanBench-XL | 19% | #5 of 9 | Official leaderboard | |
| BFCL V4 | 31.9% | #37 of 65 | Official leaderboard | |
| MCP-Bench | 0.56 | #9 of 15 | Official leaderboard | |
| SnakeBench | 21 | #114 of 188 | Official leaderboard | |
| OdysseyArena Repo System | 19.2% | #5 of 8 | Official leaderboard | |
| Berkeley Function Calling Leaderboard (BFCL) | 52.2% | #44 of 67 | Official leaderboard | |
| AdaPlanBench | 29.3% | #7 of 10 | Official leaderboard | |
| OdysseyArena AI Trading | 0.77% | #7 of 8 | Official leaderboard | |
| OdysseyArena Turn on Lights | 6.67% | #7 of 8 | Official leaderboard | |
| Galileo Agent Leaderboard v2 | 20% | #19 of 21 | Official leaderboard | |
| C3-Bench | 0% | #30 of 30 | Official leaderboard | |
| Meta Agents Research Environments (ARE) / Gaia2 | 4.40% | #13 of 13 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BRIDGE | 49.5 | #8 of 39 | Official leaderboard | |
| MedQA | 76.1% | #17 of 71 | Official leaderboard | |
| LEXam | 41.3% | #20 of 30 | Official leaderboard | |
| AssetOpsBench | 40% | #2 of 3 | Official leaderboard | |
| LEXam-hard | 26.8% | #11 of 15 | Official leaderboard | |
| MedHELM | 27.7 | #12 of 13 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| VRBench | 49.8% | #15 of 19 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AraTrust | 88.3% | #5 of 39 | Official leaderboard | |
| AthenaBench | 46.5% | #2 of 6 | Official leaderboard | |
| AutoPatchBench | 11.5% | #1 of 3 | Official leaderboard | |
| LiveSecBench | 38.9 | #22 of 34 | Official leaderboard | |
| nocot-bench (NCRI) | 62.6 | #131 of 185 | Official leaderboard | |
| Lakera B3 (Backbone Breaker Benchmark) | 46.6% | #29 of 35 | Official leaderboard | |
| Gray Swan Agent Red Teaming (ART) benchmark | 6.49% | #8 of 8 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Open LLM Leaderboard v2 | 44.9 | #2 of 99 | Official leaderboard | |
| HELM Lite | 81.2 | #13 of 71 | Official leaderboard |
Its lab's numbers put it at 127.3, number 297 of 404
The Self-Reported Index fits one score to the 8 capability results Meta reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 124.4 to 130.2. How we compute it.
Its lab's number next to a leaderboard
Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it.
Capabilities
- MMLU 86
- MMLU-Pro 68.9
- IFEval 92.1
- GPQA Diamond 50.5
- HumanEval 88.4
- MBPP 87.6
- MATH 77
- Berkeley Function Calling Leaderboard (BFCL)
- MGSM 91.1