SLA: Scaling Spatial-Language-Action model via continual pre-training

The problem

Reliable robotic manipulation requires both semantic understanding and precise control in three-dimensional space, according to the authors (arxiv.org). However, vision-language-action (VLA) foundations largely inherit visual representations from 2D vision-language pre-training, while spatial supervision is often introduced only during task-specific adaptation (arxiv.org). Developing spatial understanding as a transferable foundation-model capability therefore calls for learning from broader robot experience (arxiv.org). Typical 2D VLA foundation models inherit knowledge from visual-language models pretrained on 2D image-text pairs and learn robot control without explicitly incorporating 3D physical knowledge during pre-training, which limits their reliability in scenes that demand fine-grained spatial reasoning (arxiv.org). While several approaches use depth maps or point clouds, reliable 3D sensor data is hard to acquire because measured depth maps and point clouds can be noisy or incomplete, robotic sensors vary in types, positions, and calibration status, and portions of existing large-scale robot datasets do not include depth information (arxiv.org).

The idea

The authors present SLA, a Spatial-Language-Action Model that scales implicit spatial alignment to continual pre-training and maintains it during downstream post-training (arxiv.org). Initialized from a pretrained pi_0.5 policy, SLA jointly optimizes action prediction and alignment between intermediate visual embeddings and spatial representations from a frozen 3D perception model (arxiv.org). The authors curate and unify heterogeneous robot datasets to support spatial learning across diverse embodiments, tasks, and scenes (arxiv.org). To accommodate this diversity, they introduce feature-dependent gates that adaptively aggregate visual embeddings from multiple VLA layers for spatial supervision (arxiv.org). The teacher further exploits partially available depth and camera calibration alongside RGB observations, enabling effective use of heterogeneous geometric annotations (arxiv.org). Probabilistic state dropout encourages reliance on visual cues and mitigates overfitting to demonstration trajectories (arxiv.org). All auxiliary components are used only during training, allowing direct reuse of pretrained VLA weights and preserving the original inference architecture (arxiv.org).

The results

Extensive evaluations demonstrate that SLA achieves state-of-the-art performance on RoboDojo-Real, RoboTwin 2.0 under the Clean2Random setting, RoboCasa-Panda, and LIBERO-Plus, highlighting its strong generalization and downstream data efficiency (arxiv.org). On LIBERO-Plus, it achieves an overall success rate of 87.4% under diverse observation, language, and scene perturbations, outperforming compared baselines (arxiv.org). On RoboTwin 2.0, training on clean demonstrations yields a 72.3% success rate in the Clean2Random setting and a 79.2% average across Clean2Clean and Clean2Random (arxiv.org). On RoboCasa-Panda, SLA achieves 68.4% success with only 50 downstream demonstrations per task, exceeding compared policies trained with 300 to 3,000 demonstrations per task (arxiv.org). On RoboDojo-Real, a single policy adapted across three embodiments achieves 33.9% overall success, compared with 12.8% for pi_0.5 (arxiv.org). Across eight real-world deployment conditions, SLA further improves the average success rate over pi_0.5 by 24.0 percentage points, reaching an average of 83.8% (arxiv.org). Controlled data-scaling experiments further show that applying spatial alignment in both training stages outperforms either single-stage alternative at every tested data proportion (arxiv.org).

Why it matters

Integrating semantic reasoning with accurate physical control in the 3D real world is central to developing general-purpose robotic manipulation (arxiv.org). By incorporating 3D spatial alignment directly into continual pre-training rather than treating spatial understanding as a task-specific post-training add-on, SLA provides a scalable approach to building transferable spatial intelligence in robot foundation models (arxiv.org).

Caveats

The authors note that reliable 3D sensor data can be noisy or incomplete, and robotic sensors vary significantly in types, positions, and calibration status, introducing substantial heterogeneity into the data (arxiv.org). Furthermore, portions of existing large-scale robot datasets do not include depth information (arxiv.org). This is a preprint (arxiv.org).