← Back
AWS ML Blog

Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod

•38 min read•
#compute#deployment#nvidia#amazon
Level:Advanced
For:AI Engineers
✦TL;DR

This post details how to construct a continuous Physical AI model factory on Amazon SageMaker HyperPod using NVIDIA Cosmos 3, a Mixture‑of‑Transformers (MoT) foundation model with per‑layer joint attention and a train‑versus‑inference asymmetry. Cosmos 3’s single‑token‑stream design supports video, image, action, and sound, enabling generation, post‑training, and policy evaluation as three workloads scheduled on a shared GPU node pool, eliminating the need for separate capacity per stage. The guide includes a full end‑to‑end walkthrough of the robot‑policy stage on the public DROID dataset, with all manifests and configuration files provided in the awsome‑distributed‑ai GitHub repository. Engineers can thus achieve higher GPU goodput—useful pipeline progress per reserved GPU‑hour—by time‑sharing a persistent cluster rather than provisioning isolated pools.

⚡ Key Takeaways

  • Cosmos 3 uses a Mixture‑of‑Transformers design with per‑layer joint attention and a deliberate train‑versus‑inference asymmetry.
  • The model runs the same transformer trunk in three modes: forward‑dynamics world model, inverse‑dynamics action labeler, and deployable action policy.
  • Physical AI pipelines can share a single persistent GPU node pool on SageMaker HyperPod, improving GPU goodput versus separate stage‑specific pools.
  • The repository “awsome‑distributed‑ai” contains the Kubernetes manifests and job configurations needed to deploy each stage on Amazon EKS.
  • Cosmos 3 is released under the Linux Foundation’s OpenMDW‑1.1 license; the architecture is documented in the Cosmos 3 technical report.
💡 Why It Matters

By consolidating generation, post‑training, and policy evaluation onto one resilient GPU pool, engineers can cut infrastructure overhead and achieve higher effective throughput, directly lowering the cost per iteration of Physical AI systems such as robots or autonomous vehicles.

✅ Practical Steps

  1. Clone the awsome‑distributed‑ai GitHub repository and review the provided EKS manifests and job specifications.
  2. Deploy the manifests to an Amazon SageMaker HyperPod cluster via Amazon EKS to provision the shared multi‑terabyte storage layer and GPU node pool.
  3. Execute the robot‑policy job manifest against the public DROID dataset to run the end‑to‑end policy training and evaluation pipeline.

Want the full story? Read the original article.

Read on AWS ML Blog ↗

More like this

Introducing workload recovery and expanded infrastructure checks on Amazon SageMaker HyperPod

AWS ML Blog•#deployment

Introducing Unity Gateway Insights

Databricks Blog•#deployment

Introducing Gemini 4 Neon

Google DeepMind•#llm

DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

Hugging Face Blog•#llm

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING