Countering the 'L is Useless' Claim in VLA: Proper Positioning Boosts Instruction Generalization by 20-40%

In embodied intelligence, VLA models are often criticized for insufficient instruction generalization. In standard benchmarks like LIBERO, training and test instructions are highly similar, masking whether models truly understand task semantics. When instructions are simply rephrased, such as changing 'grab the larger red object' to 'pick up the larger red cube', model performance can drop significantly.

The paper evaluates three models on LIBERO-Para: SmolVLA achieves 72.0% success on original instructions but only 4.47% after rewriting; VLA-Adapter drops from 98.2% to 46.82%; even π0.5, trained on large-scale heterogeneous data, suffers performance loss. This indicates that high success rates do not equate to stable instruction-following ability.

Through action-level retrieval experiments, the paper finds that actions generated from rewritten instructions are still closer to the correct task than random, indicating task information is not lost. Layer-wise feature intervention further shows that in VLA-Adapter, replacing language features from rewritten instructions with original instruction features recovers 96.8% of action differences, and closed-loop success rate improves from 60% to 96%.

However, different architectures have different semantic control locations: in SmolVLA and π0.5, single-layer replacement recovers at most about 10.5% and 31.3%, indicating task information is distributed across layers. Additionally, control experiments show that dynamic images and wording changes both perturb language representations; removing the wording direction improves closed-loop success rate from 55% to 90%.

The GSR method consists of three steps: first, a frozen T5 encoder processes language instructions alone, without receiving images or states, to extract stable semantics; then, depending on the fusion location of different VLAs, the projected T5 semantics are re-injected; finally, the Action Expert is reinitialized and trained to learn new language conditioning.

On LIBERO-Para, GSR improves VLA-Adapter's success rate from 46.82% to 70.94%, SmolVLA from 4.47% to 49.12%, and π0.5 from 73.60% to 75.59%, achieving the highest PRIDE score of 70.4. All models are trained only on original instructions, without using rewritten data.

Language path authority experiments show that merely adding T5 without restructuring information flow is ineffective: VLA-Adapter's Native+T5 only improves from 46.82% to 47.31%, and SmolVLA's corresponding version is 13.49%. When the native path receives incorrect instructions, success rate drops to 5.11%, while when T5 receives incorrect instructions, it remains at 44.0%, indicating the native path still dominates control.

The experimental architecture ParaVLA uses frozen T5 and DINOv2, achieving 92% and 91% success rates on original and rewritten instructions, respectively, with only a 1 percentage point difference. However, full decoupling also brings scalability issues; future work needs to balance stable semantics, scene grounding, and model scaling capabilities.