Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod
This post details how to construct a continuous Physical AI model factory on Amazon SageMaker HyperPod using NVIDIA Cosmos 3, a Mixture‑of‑Transformers (MoT) foundation model with per‑layer joint attention and a train‑versus‑inference asymmetry. Cosmos 3’s single‑token‑stream design supports video, image, action, and sound, enabling generation, post‑training, and policy evaluation as three workloads scheduled on a shared GPU node pool, eliminating the need for separate capacity per stage. The guide includes a full end‑to‑end walkthrough of the robot‑policy stage on the public DROID dataset, with all manifests and configuration files provided in the awsome‑distributed‑ai GitHub repository. Engineers can thus achieve higher GPU goodput—useful pipeline progress per reserved GPU‑hour—by time‑sharing a persistent cluster rather than provisioning isolated pools.
⚡ Key Takeaways
- Cosmos 3 uses a Mixture‑of‑Transformers design with per‑layer joint attention and a deliberate train‑versus‑inference asymmetry.
- The model runs the same transformer trunk in three modes: forward‑dynamics world model, inverse‑dynamics action labeler, and deployable action policy.
- Physical AI pipelines can share a single persistent GPU node pool on SageMaker HyperPod, improving GPU goodput versus separate stage‑specific pools.
- The repository “awsome‑distributed‑ai” contains the Kubernetes manifests and job configurations needed to deploy each stage on Amazon EKS.
- Cosmos 3 is released under the Linux Foundation’s OpenMDW‑1.1 license; the architecture is documented in the Cosmos 3 technical report.
By consolidating generation, post‑training, and policy evaluation onto one resilient GPU pool, engineers can cut infrastructure overhead and achieve higher effective throughput, directly lowering the cost per iteration of Physical AI systems such as robots or autonomous vehicles.
✅ Practical Steps
- Clone the awsome‑distributed‑ai GitHub repository and review the provided EKS manifests and job specifications.
- Deploy the manifests to an Amazon SageMaker HyperPod cluster via Amazon EKS to provision the shared multi‑terabyte storage layer and GPU node pool.
- Execute the robot‑policy job manifest against the public DROID dataset to run the end‑to‑end policy training and evaluation pipeline.
Want the full story? Read the original article.
Read on AWS ML Blog ↗