SANA-WM: Scaling World Models to Minute-Scale with Hybrid Linear Transformers

TL;DR

NVIDIA’s researchers have unveiled SANA-WM, a 2.6B world model capable of generating one-minute 720p videos from a single image and a 6-DoF camera trajectory. Unlike current SOTA models that require multi-GPU setups and suffer from memory blowups, SANA-WM uses a Hybrid GDN-Softmax architecture and a two-stage refinement pipeline to deliver 36x higher throughput, even running on a consumer-grade RTX 5090.

The Problem: The "Memory Wall" of Minute-Scale Simulation

The transition from short video clips (5-10s) to "Minute-Scale" (60s+) world modeling is a literal and figurative computational wall. Two primary issues plague current architectures:

1. Quadratic Attention Bottleneck: Processing 720p frames for 60 seconds generates millions of tokens. Standard Softmax Attention's memory usage grows quadratically with sequence length, making single-GPU inference impossible for long horizons.
2. Trajectory Drift: Over long sequences, small errors in camera movement accumulate, causing the "world" to melt, blur, or lose its structural identity.

Methodology: The Efficiency-First Architecture

SANA-WM's performance rests on four architectural pillars that rethink how we process temporal context.

1. Hybrid Linear-Diffusion Transformer (The Backbone)

Rather than relying solely on Softmax Attention, SANA-WM uses Gated DeltaNet (GDN).

• Recurrent Context: GDN treats latent frames as steps in a recurrence, using a hidden state $S_t$ to store world information. This keeps memory usage constant regardless of video length.
• Hybridization: To prevent the model from "forgetting" the first frame or specific spatial details, every 4th block is a regular Softmax Attention layer. This provides a "global anchor" while GDN handles the temporal evolution.

2. Dual-Branch Camera Control

Precision is the core of a "World Model." SANA-WM employs two techniques to ensure the camera follows the trajectory:

• Ray-Local UCPE: Transforms world-space rays into a local basis at the attention-head level.
• Plücker Mixing: Combines 6D ray colors (direction + origin) at the fine-grained pixel level to compensate for movement inside the VAE's temporal strides.

3. Progressive Training & Two-Stage Refinement

NVIDIA advocates for a "Native Training" approach. Instead of distilling from a short-video teacher (which often lacks long-range coherence), SANA-WM was trained progressively from 5s to 60s clips. A secondary Long-Video Refiner (using LoRA-adapted flow matching) is used to polish the output, correcting structural artifacts that frequently appear in generative rollouts.

Experiments & Results: Industrial Quality on Consumer Hardware

SANA-WM was compared against baselines like HunyuanVideo and LingBot-World.

SOTA Comparison

In terms of Action Following, SANA-WM achieved a Rotation Error (RotErr) of 4.50°, significantly lower than large-scale industrial models. Its VBench Overall score of 81.89 matches models with 5x-10x more parameters.

Dramatic Efficiency Gains

• Throughput: 22.0 videos/hour (3.7x to 36x faster than competitors).
• Deployment: A distilled variant was successfully deployed on an RTX 5090. Using NVFP4 quantization, it can denoise a full 60s 720p clip in just 34 seconds.

Critical Insight: Why This Matters

SANA-WM proves that Linear Attention (GDN) is not just a theoretical toy for NLP but a viable replacement for Softmax in high-resolution visual generation. By treating video generation as a recurrent state update rather than a massive global attention problem, NVIDIA has lowered the barrier for Embodied AI and Robotics research.

However, the authors note a limitation: while the model handles camera control expertly, it still lacks explicit "3D voxel memory." This means that in highly complex revisit scenarios (returning to a room after a long time), the model relies on the recurrent hidden state alone, which may eventually drift if the sequence extends to 5 or 10 minutes.

Conclusion

SANA-WM represents a Pareto-front shift in world modeling. It offers a blueprint for creating interactive simulations that are both high-fidelity and computationally accessible, moving us closer to real-time generative game engines and robust virtual testing environments for robots.