Llama-3.1-Nemotron-Ultra-253B-v1
Llama-3.1-Nemotron-Ultra-253B-v1 places in the top three on BigO(Bench).
13
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
NVIDIA Apr 2025 Llama-3.1-Nemotron-Ultra-253B-v1
Open weights Flagship
135.9
#221 of 404
Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.
- LiveCodeBench v5 66.3% #15 /47
- BigO(Bench) 41.1% #3 /18
- AIME 2025 72.5% #106 /143
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| IFEval | 89.5% | #39 of 135 | Lab model card | |
| Arena-Hard-Auto | 87% | #15 of 39 | Lab model card | |
| JudgeBench | 79.1% | #1 of 2 | Lab model card | |
| LMArena (Elo) | 1,348 | #180 of 335 | Official leaderboard | |
| EQ-Bench 3 | 1,093 | #54 of 78 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MATH-500 | 97% | #19 of 70 | Lab model card | |
| AIME 2024 | 80.8% | #39 of 88 | Lab model card | |
| GPQA Diamond | 76% | #145 of 302 | Lab model card | |
| AIME 2025 | 72.5% | #106 of 143 | Lab model card |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BigO(Bench) | 41.1% | #3 of 18 | Official leaderboard | |
| LiveCodeBench v5 | 66.3% | #15 of 47 | Lab model card |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Berkeley Function Calling Leaderboard (BFCL) | 56.7% | #35 of 67 | Official leaderboard | |
| BFCL V4 | 10% | #64 of 65 | Official leaderboard |
Its lab's numbers put it at 135.9, number 221 of 404
The Self-Reported Index fits one score to the 7 capability results NVIDIA reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 133.5 to 138.3. How we compute it.
- MATH-500 97
- IFEval 89.5
- Arena-Hard-Auto 87
- AIME 2024 80.8
- GPQA Diamond 76
- AIME 2025 72.5
- LiveCodeBench v5 66.3
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 2 of them aren't in the launch post.
Llama-Nemotron: Efficient Reasoning Models May 2, 2025
Capabilities
- GPQA Diamond 76
- AIME 2024 80.8
- AIME 2025 72.5
- MATH-500 97
- Berkeley Function Calling Leaderboard (BFCL)
- LiveCodeBench v5 66.3
- IFEval 88.9
- Arena-Hard-Auto 87
- JudgeBench 79.1