Model Drop

Llama 3.1 Nemotron Ultra

April 7, 2025 2 min read

Llama 3.1 Nemotron Ultra, which NVIDIA released as open weights on April 7, 2025 after announcing the Llama Nemotron family at GTC on March 18, is a 253B reasoning model compressed from Meta's Llama 3.1 405B that can switch its step-by-step thinking on or off.

Model: Llama 3.1 Nemotron Ultra (nvidia/Llama-3_1-Nemotron-Ultra-253B-v1)
Maker: NVIDIA
Ship date: April 7, 2025
Pricing: Free (open weights)
Context: 128K tokens
Where you can use it:
- Hugging Face weights
- build.nvidia.com NIM from April 10
When to reach for it:
- Open reasoning on a single 8xH100 node
- Agents that lean on tool calls and RAG
- Skip it for images; it's text only
How it compares:
- Under half the size of DeepSeek R1
- Text only, unlike Llama 4 Maverick
What makes it different:
- Reasoning toggled on or off by system prompt
- Shrunk from Llama 3.1 405B by NAS
Early reactions: First-week Hugging Face threads asked for AWQ, GPTQ, FP8 and FP4 quantized builds and how to reproduce NVIDIA's benchmark scores.
Read more: NVIDIA Llama Nemotron Ultra Open Model Delivers Groundbreaking Reasoning Accuracy, Hugging Face model card, Llama-Nemotron: Efficient Reasoning Models

The benchmarks

Scores are Ultra with reasoning on (temperature 0.6, top-p 0.95, 32K context, averaged over up to 16 runs) from NVIDIA's launch chart, against DeepSeek R1, Llama 4 Maverick and Llama 3.1 405B; most rows show only one or two of those rivals, and the chart's unreleased Llama 4 Behemoth bars are left out.

NVIDIA's numbers

GPQA Diamond (PhD-level science, no tools): 76.0%. +4.5 over DeepSeek R1 (71.5%), +6.2 over Llama 4 Maverick (69.8%), +32.6 over Llama 3.1 405B (43.4%).
AIME 2025 (competition math, no tools): 72.5%. +2.5 over DeepSeek R1 (70.0%).
AIME 2024 (competition math, no tools): 80.8%. +1.0 over DeepSeek R1 (79.8%).
LiveCodeBench (Aug 2024 to Feb 2025 problems): 66.3%. +0.4 over DeepSeek R1 (65.9%).
BFCL V2 Live (function calling): 74.1%. +15.4 over Llama 3.1 405B (58.7%).
MATH-500 (competition math problems): 97.0%. −0.3 under DeepSeek R1 (97.3%), +30.8 over Llama 3.1 405B (66.2%).
Arena Hard (chat preference): 87.0%. −5.0 under DeepSeek R1 (92.0%), +20.8 over Llama 3.1 405B (66.2%).

Independent numbers

Artificial Analysis Intelligence Index (current v4.3.2, AA estimate): 7.5. −3.9 under DeepSeek R1 (11.4), −1.8 under Llama 4 Maverick (9.3), +0.2 over Llama 3.1 405B (7.3).