Summary
LiLa-WAM proposes a compact world-action model that reasons about the future in a shared latent space jointly shaped by future-state prediction and action generation, avoiding both the heavy pixel-space video generation of many WAMs and the multi-stage training pipelines of prior latent-space methods. The full model is 0.5B parameters (0.2B trainable) and trains end-to-end on a single 24GB GPU.
Key Contributions
- Single-stage joint training of a compact latent reasoning space for both prediction and control.
- Removes the need for pixel-accurate video rollouts at train or inference time.
- Strong compute-efficiency claim: trainable on a single consumer/prosumer GPU.
Strengths
- Very low compute budget for a WAM-class model (~110 GPU-hours on one RTX 5090 for all 50 RoboTwin 2.0 tasks jointly).
- Strong reported RoboTwin 2.0 benchmark numbers: 90.48% average success across 50 tasks under the clean setting.
- Open-sourced code.
Weaknesses
- Benchmarked mainly in simulation (RoboTwin 2.0); real-robot generalization is untested.
- The latent-only representation may sacrifice fine visual detail useful for some contact-rich tasks.
Open Questions
- How does LiLa-WAM scale to longer horizons or additional embodiments?
- What is its real-world sim-to-real transfer performance?
Significance
Continues the efficiency thread already well-represented in this vault’s WAM coverage (Flash-WAM, Efficient-WAM), pushing training cost down to a single consumer GPU while maintaining strong simulated benchmark results.