With Groq 3 LPX in Full Production, NVIDIA Extends Vera Rubin Inference for Agents
NVIDIA announced that its Vera Rubin NVL72 system, now paired with the Groq 3 LPX accelerator, can generate 3,400 output tokens per second on a 100,000‑token long‑context run of the open‑source agentic model Gemma 4 31B—four times faster than the closest competing platform. The full‑stack, “extreme codesign” approach co‑optimizes compute, networking (Spectrum‑X Ethernet) and inference acceleration to meet the throughput and latency demands of agentic AI workloads. Engineers can leverage this rack‑scale solution to lower token‑costs and achieve ultra‑low latency in multi‑agent reasoning pipelines.
⚡ Key Takeaways
- 3,400 output tokens / s achieved on Gemma 4 31B for 100,000‑token contexts.
- Architecture combines NVIDIA Vera Rubin NVL72, Groq 3 LPX accelerator, and Spectrum‑X Ethernet for unified stack.
- Performance gain: 4× faster token generation than the nearest alternative platform.
- Integration path: add Groq 3 LPX to an existing Vera Rubin NVL72 rack to unlock the speed boost.
- Prerequisite: deployment on a Vera Rubin NVL72 rack‑scale system; the accelerator is not a standalone chip.
The token‑generation speed and cost reductions enable production‑grade agentic AI services to scale to longer contexts and higher interaction rates without prohibitive latency or expense.
✅ Practical Steps
- Deploy NVIDIA Groq 3 LPX alongside an existing Vera Rubin NVL72 rack to enable the 3,400 t/s token rate.
- Connect the racks using NVIDIA Spectrum‑X Ethernet to maintain high‑bandwidth, lossless data flows for multi‑agent workloads.
- Validate performance with the Artificial Analysis benchmark running Gemma 4 31B on 100,000‑token contexts.
Want the full story? Read the original article.
Read on NVIDIA Blog ↗