← Back
AWS ML Blog

Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKS

•8 min read•
#agents#amazon#nvidia#inference
✦TL;DR

NVIDIA’s Nemotron 3.5 Super VL, a 120‑B parameter mixture‑of‑experts model, merges multimodal perception, reasoning, and tool‑use capabilities into a single architecture. The post details how the model’s expert routing and dynamic sparsity enable efficient scaling on GPU‑rich clusters, and demonstrates end‑to‑end deployment on Amazon EKS, EKS Auto Mode, and SageMaker HyperPod EKS using NVIDIA Dynamo. Engineers can leverage the Dynamo SDK to spin up a 120‑B inference workload that automatically balances load across expert shards, while the deployment pipeline supports rapid scaling and cost‑effective GPU utilization. The approach trades off a modest increase in per‑request latency for significant reductions in GPU memory footprint and overall inference cost.

⚡ Key Takeaways

  • Nemotron 3.

Want the full story? Read the original article.

Read on AWS ML Blog ↗

More like this

Introducing workload recovery and expanded infrastructure checks on Amazon SageMaker HyperPod

AWS ML Blog•#deployment

Introducing Gemini 4 Neon

Google DeepMind•#llm

DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

Hugging Face Blog•#llm

Mathematical proof, powered by AI. And no hallucinations!

Amazon Science•#agents

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING