NVIDIA Vera Rubin NVL72 Delivers Leading Performance in MLPerf Inference v6.1 Debut
NVIDIA’s new Vera Rubin NVL72 system debuted in MLPerf Inference v6.1, delivering up to 3.7× higher throughput than the prior‑generation GB300 NVL72 on the Qwen3‑VL benchmark and up to 2.5× higher on DeepSeek‑R1. The gains stem from enhanced Tensor Cores, the Transformer Engine, NVFP4 precision, and software stacks such as vLLM with NVIDIA Dynamo and TensorRT‑LLM. For engineers, the result translates into more tokens per rack, higher user capacity, and lower cost‑per‑token when scaling inference workloads.
⚡ Key Takeaways
- Up to 3.7× higher throughput on Qwen3‑VL (offline, server, interactive) versus GB300 NVL72.
- Uses vLLM with the NVIDIA Dynamo open‑source inference framework for Qwen3‑VL.
- Uses NVIDIA TensorRT‑LLM library for DeepSeek‑R1, achieving up to 2.5× higher throughput.
- NVL72’s enhanced Tensor Cores and Transformer Engine accelerate both prefill and decode stages, while NVFP4 precision cuts memory footprint with minimal quality loss.
- Disaggregated serving (separating prefill and decode) and large‑scale expert parallelism are employed to maximize mixture‑of‑experts efficiency.
The performance leap lets production teams serve more requests per GPU rack and reduce inference cost, directly improving revenue potential for token‑based AI services.
✅ Practical Steps
- Deploy the Vera Rubin NVL72 system and run inference workloads with vLLM configured to use the NVIDIA Dynamo framework for models like Qwen3‑VL.
- For DeepSeek‑R1 or similar MoE models, integrate NVIDIA TensorRT‑LLM to exploit the NVL72’s Tensor Cores and NVFP4 precision.
- Structure your serving pipeline to separate prefill and decode stages, leveraging disaggregated serving and expert parallelism as demonstrated in the MLPerf submissions.
Want the full story? Read the original article.
Read on NVIDIA Blog ↗