Summary
Lift3D-VLA is a unified VLA framework that equips 2D pretrained vision-language models with explicit 3D point-cloud reasoning and temporally coherent action generation. It lifts standard 2D backbones into 3D through a geometric alignment step and a dual-objective self-supervised pretraining scheme that jointly reconstructs current geometry and predicts future geometric evolution.
Key Contributions
- Enhanced 2D model-lifting strategy that geometrically aligns 3D points with pretrained 2D positional embeddings, enabling 3D priors without full retraining
- Geometry-Centric Masked Autoencoding (GC-MAE): dual-objective SSL reconstructing the current point cloud while predicting its future geometric state, grounding the vision encoder in both structure and dynamics
- Layer-wise temporal action modeling that leverages multiple LLM layers to collaboratively predict action chunks, producing temporally consistent trajectories
Significance
Achieves 10.8% and 11.1% higher mean success rates on MetaWorld and RLBench respectively versus best prior VLA methods across 22 simulated and 8 real-world manipulation tasks, establishing a new state-of-the-art for 3D-aware VLA manipulation.