← Back
Hugging Face Blog

New NVIDIA Cosmos 3 Open Models for Interactive World Simulation

•7 min read•
#nvidia#inference
New NVIDIA Cosmos 3 Open Models for Interactive World Simulation
Level:Advanced
For:AI Engineers
✦TL;DR

NVIDIA introduced four new open‑weight checkpoints in the Cosmos 3 Nano family, each with 16 B trainable parameters and licensed under OpenMDW 1.1. The models—Cosmos3‑Nano‑Transfer‑Auto, Cosmos3‑Nano‑Sim‑Auto, Cosmos3‑Nano‑Sim‑Transfer, and Cosmos3‑Nano‑Sim‑Bimanual—support multi‑view, multi‑sensor driving data generation, synchronized camera/LiDAR streaming, text‑to‑video simulation, and robot‑action video prediction respectively. These checkpoints enable researchers to build interactive world simulations for robotics and autonomous‑driving research, with the ability to fine‑tune on custom sensor rigs and domains. Engineers can now integrate open‑weight world models directly into their pipelines for offline data synthesis or causal streaming simulation.

⚡ Key Takeaways

  • All four checkpoints are post‑trained variants of Cosmos 3 Nano with 16 B trainable parameters.
  • Cosmos3‑Nano‑Transfer‑Auto generates synchronized video from up to 11 camera views and joint RGB‑LiDAR data using per‑camera World Scenario Map controls and optional text prompts.
  • Cosmos3‑Nano‑Sim‑Auto provides causally distilled streaming generation of camera and LiDAR observations from driving controls.
  • Cosmos3‑Nano‑Sim‑Transfer converts simulator controls and a text prompt into photorealistic video.
  • Cosmos3‑Nano‑Sim‑Bimanual predicts video conditioned on robot or camera actions.
💡 Why It Matters

By releasing open‑weight, multi‑sensor world models, NVIDIA gives engineers a ready‑to‑use foundation for creating high‑fidelity simulated environments, accelerating data generation and policy evaluation for autonomous‑driving and robotics systems.

✅ Practical Steps

  1. Download the desired checkpoint from the provided link and run inference with Cosmos3‑Nano‑Transfer‑Auto for offline multi‑sensor data synthesis.
  2. Post‑train Cosmos3‑Nano‑Transfer‑Auto on your own driving dataset to adapt to your specific camera layout and LiDAR configuration before using it for offline generation or streaming simulation.
  3. Use the causally distilled Cosmos3‑Nano‑Sim‑Auto checkpoint to stream synchronized observations directly from driving control inputs for interactive simulation loops.

Want the full story? Read the original article.

Read on Hugging Face Blog ↗

More like this

Helping AI text detectors keep up with changing conditions

MIT News AI•#llm

Hugging Face and OpenRouter join forces to make open-weight models easy to run

Hugging Face Blog•#inference

Activation and Floating-Point Quantization

Machine Learning Mastery•#inference

New research could lead to faster, trustworthy AI models for climate prediction

MIT News AI•#inference

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING