Llama-3.3-Nemotron-Super-49B-v1
Llama-3.3-Nemotron-Super-49B-v1 places in the top three on MBPP.
14
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
NVIDIA Mar 2025 Llama-3.3-Nemotron-Super-49B-v1
Open weights Mid-size
130.5
#270 of 404
Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.
- LiveCodeBench v5 45.5% #38 /47
- Misguided Attention 57.4% #8 /45
- SnorkelSequences 44.8% #17 /25
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MT-Bench | 9.17 | #4 of 23 | Lab model card | |
| IFEval | 89.2% | #42 of 135 | Lab model card | |
| Arena-Hard-Auto | 88.3% | #13 of 39 | Lab model card | |
| LMArena (Elo) | 1,328 | #199 of 335 | Official leaderboard | |
| JudgeBench | 69.7% | #2 of 2 | Lab model card |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Misguided Attention | 57.4% | #8 of 45 | Official leaderboard | |
| MATH-500 | 96.6% | #25 of 70 | Lab model card | |
| GPQA Diamond | 66.7% | #195 of 302 | Lab model card | |
| SnorkelSequences | 44.8% | #17 of 25 | Official leaderboard | |
| SnorkelGraph | 29% | #17 of 24 | Official leaderboard | |
| AIME 2024 | 67.5% | #69 of 88 | Lab model card | |
| AIME 2025 | 60% | #128 of 143 | Lab model card |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MBPP | 91.3% | #2 of 71 | Lab model card | |
| LiveCodeBench v5 | 45.5% | #38 of 47 | Lab model card |
Its lab's numbers put it at 130.5, number 270 of 404
The Self-Reported Index fits one score to the 8 capability results NVIDIA reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 128.3 to 132.6. How we compute it.
- MATH-500 96.6
- MBPP 91.3
- IFEval 89.2
- Arena-Hard-Auto 88.3
- AIME 2024 67.5
- GPQA Diamond 66.7
- AIME 2025 60
- LiveCodeBench v5 45.5
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 3 of them aren't in the launch post.
Llama-Nemotron: Efficient Reasoning Models May 2, 2025
Capabilities
- GPQA Diamond 66.7
- AIME 2024 67.5
- AIME 2025 60
- MATH-500 96.6
- Berkeley Function Calling Leaderboard (BFCL)
- LiveCodeBench v5 45.5
- IFEval 89.2
- Arena-Hard-Auto 88.3
- JudgeBench 69.7