← Back
NVIDIA Blog

NVIDIA Vera Rubin NVL72 Delivers Leading Performance in MLPerf Inference v6.1 Debut

•5 min read•
#inference#nvidia#compute
Level:Advanced
For:AI Engineers
✦TL;DR

NVIDIA’s new Vera Rubin NVL72 system debuted in MLPerf Inference v6.1, delivering up to 3.7× higher throughput than the prior‑generation GB300 NVL72 on the Qwen3‑VL benchmark and up to 2.5× higher on DeepSeek‑R1. The gains stem from enhanced Tensor Cores, the Transformer Engine, NVFP4 precision, and software stacks such as vLLM with NVIDIA Dynamo and TensorRT‑LLM. For engineers, the result translates into more tokens per rack, higher user capacity, and lower cost‑per‑token when scaling inference workloads.

⚡ Key Takeaways

  • Up to 3.7× higher throughput on Qwen3‑VL (offline, server, interactive) versus GB300 NVL72.
  • Uses vLLM with the NVIDIA Dynamo open‑source inference framework for Qwen3‑VL.
  • Uses NVIDIA TensorRT‑LLM library for DeepSeek‑R1, achieving up to 2.5× higher throughput.
  • NVL72’s enhanced Tensor Cores and Transformer Engine accelerate both prefill and decode stages, while NVFP4 precision cuts memory footprint with minimal quality loss.
  • Disaggregated serving (separating prefill and decode) and large‑scale expert parallelism are employed to maximize mixture‑of‑experts efficiency.
💡 Why It Matters

The performance leap lets production teams serve more requests per GPU rack and reduce inference cost, directly improving revenue potential for token‑based AI services.

✅ Practical Steps

  1. Deploy the Vera Rubin NVL72 system and run inference workloads with vLLM configured to use the NVIDIA Dynamo framework for models like Qwen3‑VL.
  2. For DeepSeek‑R1 or similar MoE models, integrate NVIDIA TensorRT‑LLM to exploit the NVL72’s Tensor Cores and NVFP4 precision.
  3. Structure your serving pipeline to separate prefill and decode stages, leveraging disaggregated serving and expert parallelism as demonstrated in the MLPerf submissions.

Want the full story? Read the original article.

Read on NVIDIA Blog ↗

More like this

AI’s GPU focus meets a CPU reality check at Dell and AMD

SiliconANGLE AI•#agents

NVIDIA RTX Spark Compact Desktops Out Now

NVIDIA Blog•#compute

Helping AI text detectors keep up with changing conditions

MIT News AI•#llm

Hugging Face and OpenRouter join forces to make open-weight models easy to run

Hugging Face Blog•#inference

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING