Back to feed
News Story
BStandard51
NVIDIA Developer Blog
1 sources

Beyond VLAs: How World Action Models Reshape Robot Manipulation

This article discusses the key challenge of generalization in robot manipulation and proposes the concept of World Action Models, which go beyond Vision-Language-Action models (VLAs). These models aim to improve policy generalization under changes in object shape, position, or lighting by understanding the underlying physics of tasks rather than mimicking demonstrations. The article emphasizes the importance of world models as the backbone of robot policies.

SynthePulse Insight · AI deep readingMembers

World Action Models: A Robot Policy Revolution from 'Describing the World' to 'Predicting the World'

Version 1 · 1 source

While vision-language models (VLMs) excel at describing the world but fail to predict its evolution, world action models (WAMs), built on video world models, bring physical generalization to robot policies, enabling zero-shot transfer. NVIDIA Cosmos 3, as an open foundation model, is driving this shift from research to multi-tier deployment.

  • WAMs are built on video world models, not VLMs, enabling them to learn physical dynamics and achieve zero-shot transfer to new tasks, robots, and environments.
  • NVIDIA Cosmos 3 uses a Mixture-of-Transformers architecture, trained on approximately 767 million images, 348 million videos, and 8 million action samples.
  • The Cosmos 3-based DROID policy improved RoboLab success rates from 28.1% to 36.8%, demonstrating the gains from physical priors.
Open section navigationLimitations of VLA and the Rise of WAM

Limitations of VLA and the Rise of WAM

Traditional robot policies rely on pretrained vision-language models (VLMs) as backbones, forming vision-language-action (VLA) models. VLMs are optimized to generate image descriptions, not to predict scene evolution, so VLAs perform well in semantic generalization but fall short in physical generalization—when object shapes, positions, or lighting change, the policy may fail.

World action models (WAMs) replace the backbone with a video world model, directly modeling how the world evolves. NVIDIA researcher Jim Fan proposed in a talk that 'VLAs are dead, long live world action models,' a view summarized as an industry trend. The core advantage of WAMs is that post-training does not require learning dynamics from scratch; instead, it specializes based on existing physical priors.

Free for now

Read the full analysis

4 more sections of analysis, plus the full takeaway

Loading

Credibility boundary

This article's information primarily comes from NVIDIA's official technical blog, which is vendor analysis. Some data (such as RoboLab success rates) comes from their technical report and has not been independently verified. Statements like 'VLAs are dead' are opinions, not facts.

Sources for this version

  1. Beyond VLAs: How World Action Models Reshape Robot ManipulationNVIDIA Developer Blog

Primary report

NVIDIA Developer Blog

Primary source