NVIDIA Cosmos 3 Open-Sourced: The World Model Installed on Edge Devices for the First Time

NVIDIA has uploaded the Cosmos 3 weights to Hugging Face. Early this morning, NVIDIA officially opened downloads for the Cosmos 3 series — including two main variants, Super with 64.6 billion parameters and Nano with 15.7 billion — while also releasing an Edge branch specifically optimized for on‑device deployment. This is the most significant update on NVIDIA’s physical AI world‑model line since Cosmos 2 earlier this year, and it comes with fully open weights.

If you haven’t caught up with the world‑model wave — in short, it’s not the same as an LLM. LLMs model language distributions, while world models capture the dynamics of the physical world: how a cup falls off a table when pushed, how a car skids during hard braking on a rainy day, how a robotic gripper deforms at the contact surface. Previously, these capabilities were scattered across various simulators and task‑specific models; Cosmos 3 aims to provide a unified backbone.

Omnimodal, not just multimodal

In the official paper, Cosmos 3 is described as an “Omnimodal World Model.” This goes beyond multimodal — it must not only understand text, images, video, and audio, but also action sequences. The overall architecture is based on a mixture‑of‑transformers (MoT): different modalities take different expert branches, yet share a unified inference core.

This design yields several direct outcomes:

• Text‑to‑image and image‑to‑video aren’t its main selling points — but it still reaches SOTA. According to Artificial Analysis benchmarks, the post‑trained Cosmos 3 currently ranks as the strongest open‑weights text‑to‑image and image‑to‑video model. A respectable side product.
• Strongest policy model on RoboArena. This is NVIDIA’s real focus — as the policy backbone for embodied agents, Cosmos 3 surpasses all prior open‑source solutions on robotic manipulation tasks.
• Action‑conditioned generation. Given a video plus an action sequence (for example, a 6‑DoF end‑effector trajectory), the model predicts the next few seconds of visual frames. This is the concept of a “world model as simulator” — robot policies can perform rollout and planning inside the model without physically breaking an actual cup.

Edge edition: a world model that fits on‑device for the first time

The real highlight is Cosmos 3 Edge. Super and Nano target data centers, while Edge is the first time NVIDIA has compressed a world model to run on Jetson Thor and consumer RTX‑class hardware.

For robotics and autonomous‑driving teams, this matters far more than a mere parameter‑count reduction:

1. Latency. Running a 60 B world model in the cloud can take hundreds of ms per rollout — unacceptable for control loops. The Edge version targets 30 fps‑class inference, aligned with vision‑servo feedback.
2. Data privacy. In industrial settings, factory‑floor video must stay on‑prem. Previously you either gave up on world models or distilled a small private one.
3. Cost. You can’t equip every deployed robot with an H100 GPU.

The Edge version uses NVIDIA’s own quantization + distillation pipeline. The official blog notes that on Jetson AGX Thor it achieves near‑real‑time image‑to‑video inference. Resolution can’t match the Super model, but it’s sufficient for short‑horizon action prediction.

The dataset is released too

Unusually, NVIDIA also opened part of the physical‑AI training dataset, including:

• Multi‑view video + action‑trajectory pairs for robotic pick‑and‑place scenes
• Long‑tail driving scenarios (rare events: extreme weather, sudden cut‑ins, irregular obstacles)
• Supervised image‑to‑video data for warehouse‑safety scenes
• Ambient‑sound data for audio generation

The data is balanced across modalities and task types to avoid overfitting narrow domains — a key detail, since many open‑source world models over the past year have failed due to biased datasets that only learn to produce driving footage while breaking on robotics tasks.

Getting started: Diffusers has native support

Hugging Face has already integrated Cosmos 3 into Diffusers; install the nightly build to run:

pip install -U git+https://github.com/huggingface/diffusers
pip install transformers accelerate

A minimal image‑to‑video example with Nano looks like this:

from diffusers import Cosmos3Pipeline
import torch

pipe = Cosmos3Pipeline.from_pretrained(
    "nvidia/Cosmos3-Nano",
    torch_dtype=torch.bfloat16,
).to("cuda")

video = pipe(
    image=input_image,
    prompt="robotic arm picks up the red cube and places it in the bin",
    num_frames=49,
    guidance_scale=6.0,
).frames[0]

For action‑conditioned generation, provide an additional action_trajectory tensor — its format (standard 7‑DoF joint angles or end‑effector pose sequences) is detailed in the model card.

Teams preferring the official Cosmos framework instead of Diffusers can find full inference scripts in the nvidia/Cosmos3‑Super model card, including best practices for distributed deployment and KV‑cache reuse.

Position among competitors

The world‑model track has been lively over the past six months. Google DeepMind has Genie 3, Meta has V‑JEPA 2, and several Chinese labs and startups have their own releases. Placing Cosmos 3 side‑by‑side:

• Vs Genie 3: Genie focuses on interactive world generation, letting users walk around generated worlds like a game. Cosmos 3 is more pragmatic, targeting robotics and autonomous‑driving downstream tasks rather than pure interactivity.
• Vs V‑JEPA 2: The JEPA series follows a representation‑learning path without pixel‑level generation. Cosmos 3 tackles both: it produces embeddings for policy training and generative outputs for data augmentation.
• Vs closed‑source systems: Tesla and Waymo each have internal world models but keep them private. Cosmos 3 is currently the closest open‑weights model to industrial‑grade capability.

A reality check: achieving physical consistency remains difficult. Cosmos 3’s videos look convincing over 1–2 seconds, but beyond 5 seconds issues appear — object conservation and rigid‑body constraints break down. Cups may duplicate, robot joints bend oddly. This isn’t unique to NVIDIA; it’s an open problem for the entire field.

Who should pay attention

By category:

• Embodied‑AI teams: Cosmos 3 Super can serve directly as a policy backbone for fine‑tuning, while Edge can run on real robots for real‑time prediction — the most promising open‑source starting point today.
• Autonomous‑driving simulation teams: The long‑tail‑scenario dataset alone is valuable; the model can generate corner‑case data far cheaper than handcrafted CARLA scenes.
• Video‑generation teams: If you care only about text‑to‑video or image‑to‑video, the post‑trained Cosmos 3 Super currently sets open‑source SOTA and can drop into existing pipelines.
• LLM‑application teams: For now you can watch from afar. Integration of world models and LLMs is still early‑stage, but worth following — OpenAI’s Sora 2 and Anthropic’s recent leaks both move in this direction.

Practical caveats

From today’s testing:

1. Super inference requires at least 2 × H100 (80 GB); single‑card memory is insufficient for bf16 weights. Nano barely fits on one A100 40 GB.
2. On Jetson, Edge has a long initial load (~2 min) due to TensorRT engine compilation; it caches afterward.
3. The Diffusers integration currently supports only the video‑generation path; action generation still needs the official Cosmos framework.
4. License: NVIDIA Open Model License — check commercial terms carefully; it’s not equivalent to Apache 2.0.

Final thoughts

World models have been hot since last year, but few have reached practical deployment. With Cosmos 3’s open release — especially the Edge edition — the technology moves from “paper and demo” into “runs on a real robot.”

For developers, it’s a solid entry point. If hardware is limited, start with Nano to learn the architecture and data formats, then move to Super or distill your own for production.

NVIDIA’s move also signals something larger: the physical‑AI arms race has begun. Jensen Huang’s line “the next wave of AI is Physical AI” is no longer just a keynote slogan — it now comes with downloadable weights.

References

• Cosmos 3 Official Blog – Hugging Face – Technical overview co‑authored by NVIDIA and Hugging Face, with capability demos and Diffusers examples
• Cosmos 3 Edge Release Blog – Details on Edge architecture and on‑device deployment
• Cosmos 3 Paper Page – Omnimodal World Models technical report describing the mixture‑of‑transformers design
• nvidia/Cosmos3‑Super Model Card – Super‑version weight downloads and inference scripts