← Back
NVIDIA Blog

With Groq 3 LPX in Full Production, NVIDIA Extends Vera Rubin Inference for Agents

•10 min read•
#agents#inference#nvidia
Level:Advanced
For:AI Engineers
✦TL;DR

NVIDIA announced that its Vera Rubin NVL72 system, now paired with the Groq 3 LPX accelerator, can generate 3,400 output tokens per second on a 100,000‑token long‑context run of the open‑source agentic model Gemma 4 31B—four times faster than the closest competing platform. The full‑stack, “extreme codesign” approach co‑optimizes compute, networking (Spectrum‑X Ethernet) and inference acceleration to meet the throughput and latency demands of agentic AI workloads. Engineers can leverage this rack‑scale solution to lower token‑costs and achieve ultra‑low latency in multi‑agent reasoning pipelines.

⚡ Key Takeaways

  • 3,400 output tokens / s achieved on Gemma 4 31B for 100,000‑token contexts.
  • Architecture combines NVIDIA Vera Rubin NVL72, Groq 3 LPX accelerator, and Spectrum‑X Ethernet for unified stack.
  • Performance gain: 4× faster token generation than the nearest alternative platform.
  • Integration path: add Groq 3 LPX to an existing Vera Rubin NVL72 rack to unlock the speed boost.
  • Prerequisite: deployment on a Vera Rubin NVL72 rack‑scale system; the accelerator is not a standalone chip.
💡 Why It Matters

The token‑generation speed and cost reductions enable production‑grade agentic AI services to scale to longer contexts and higher interaction rates without prohibitive latency or expense.

✅ Practical Steps

  1. Deploy NVIDIA Groq 3 LPX alongside an existing Vera Rubin NVL72 rack to enable the 3,400 t/s token rate.
  2. Connect the racks using NVIDIA Spectrum‑X Ethernet to maintain high‑bandwidth, lossless data flows for multi‑agent workloads.
  3. Validate performance with the Artificial Analysis benchmark running Gemma 4 31B on 100,000‑token contexts.

Want the full story? Read the original article.

Read on NVIDIA Blog ↗

More like this

Introducing Gemini 4 Neon

Google DeepMind•#llm

DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

Hugging Face Blog•#llm

Mathematical proof, powered by AI. And no hallucinations!

Amazon Science•#agents

AI’s GPU focus meets a CPU reality check at Dell and AMD

SiliconANGLE AI•#agents

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING