Summary
This paper proposes Selected Diffusion Noise (SDN), a training-free test-time method that treats the initial noise vector of a diffusion-based VLA policy as a controllable degree of freedom to improve robustness to spurious visual correlations and reduce action jitter. It targets a well-known brittleness of diffusion-based manipulation policies (e.g., π0, GR00T-N1.5/1.6): they generalize well on average but can latch onto spurious visual cues and produce noisy, unstable action trajectories under perturbation.
Key Contributions
- Identifies the diffusion noise space itself (rather than model weights or inputs) as an exploitable lever for robustness at test time, requiring no fine-tuning or architecture changes.
- A dual-objective noise selection scheme: (1) sample noise vectors maximally separated from a reference set to reduce reliance on spurious correlations, and (2) select candidates yielding more temporally coherent action trajectories to reduce jitter.
- Demonstrates the method is policy-agnostic, tested across π0, GRoot-N1.5, and GRoot-N1.6 on two simulation benchmarks (Google Robot, WidowX) and two real-world robot datasets.
- Shows improved robustness specifically under object-masked / occluded observations, a proxy for spurious-correlation stress-testing.
Strengths
- Training-free and model-agnostic — can be layered onto already-deployed diffusion VLA policies with no retraining, which is operationally attractive.
- Evaluates across multiple backbones and both sim and real settings rather than a single narrow benchmark, giving more confidence the effect isn’t backbone-specific.
- Addresses two related but distinct failure modes (spurious correlation reliance and action jitter) with a single unified mechanism, which is a nice unification if it holds.
Weaknesses
- “Maximally separated from a reference set” noise sampling implies extra inference-time computation (search/scoring over candidate noise vectors); the added latency/compute cost versus standard single-pass denoising isn’t clearly quantified in the summarized results.
- Object-masking is a reasonable but somewhat artificial proxy for real spurious correlation; it’s unclear how well gains transfer to more naturalistic distribution shifts (background clutter, lighting, novel object textures) that weren’t explicitly masked.
- As a test-time-only intervention, it cannot fix cases where the underlying policy has no signal at all to distinguish causal from spurious cues — it can only reweight among noise-conditioned outputs the model is already capable of producing.
Open Questions
- How does SDN’s benefit scale with the severity of distribution shift — does it help mainly at mild perturbation levels, or also under severe occlusion/OOD conditions?
- Is there a principled way to choose the “reference set” for noise separation, and how sensitive are results to that choice?
- Could SDN be combined with training-time robustness interventions (e.g., data augmentation, causal regularization) for compounding gains, or is its benefit redundant with those approaches?
Significance
The paper is a useful addition to the growing toolkit of training-free test-time interventions for diffusion/flow-based VLA policies, showing that the noise-sampling process — usually treated as an implementation detail — can be a meaningful, cheap-to-adopt lever for robustness against spurious correlations.