NVIDIA Nemotron 3.5 Super VL is now available in Amazon SageMaker JumpStart
NVIDIA Nemotron 3.5 Super VL, a 120 B‑parameter vision‑language model with a 12 B‑active‑parameter MoE language backbone and C‑RADIO vision encoder, is now available through Amazon SageMaker JumpStart. The model supports up to 1 M token context, includes a built‑in Multi‑Token Prediction head for speculative decoding, and is offered in two variants: a quantized NVFP4 (4‑bit W4A4 on Blackwell GPUs, ~75 GB checkpoint) and a full‑precision BF16 reference (~249 GB). Benchmarks show the NVFP4 variant delivers optimized latency and throughput on SageMaker instances such as ml.g7.24xlarge, while the BF16 version is intended for further fine‑tuning or custom quantization. Engineers can deploy the model with a few lines of the SageMaker Python SDK, enabling multimodal applications like document intelligence and video QA.
⚡ Key Takeaways
- NVFP4 variant runs with 4‑bit weights and activations (W4A4) on Blackwell GPUs, checkpoint size ≈ 75 GB.
- Architecture combines 88‑layer hybrid backbone (interleaved Mamba‑2 and MoE layers, 512 routed experts, 22 active per token) with a C‑RADIO vision encoder.
- Speculative decoding is enabled via the built‑in MTP head, but not activated in the default JumpStart serving configuration.
- Deploy via SageMaker Python SDK or SageMaker Studio using ml.g7.24xlarge (default) or ml.g7.48xlarge instances.
- Full‑precision BF16 checkpoint (~249 GB) is provided for customization (SFT, RL, domain adaptation) but lacks the NVFP4 inference optimizations.
The availability of a high‑capacity, multimodal MoE model with built‑in speculative decoding on SageMaker lets production teams add vision‑language capabilities without managing custom infrastructure, while the quantized NVFP4 variant offers cost‑effective inference at scale.
✅ Practical Steps
- Use the SageMaker Python SDK to instantiate the Nemotron 3.5 Super VL model on a ml.g7.24xlarge instance, selecting the NVFP4 variant for optimized inference.
- Configure the request payload to include images, video frames, or text and set the “reasoning” flag as needed (enabled by default).
- If further fine‑tuning is required, switch to the BF16 checkpoint (~249 GB) and apply your own SFT or RL pipelines.
Want the full story? Read the original article.
Read on AWS ML Blog ↗