StreamWAM is a research framework for streaming World-Action Models (WAMs). It provides a unified testbed for systematically studying and comparing different streaming strategies for WAM-based robot control.
Building on this framework, we introduce StreamWAM, an action-conditioned streaming formulation that feeds the prefix of actions currently being executed by the robot back into the world model. This explicitly conditions future video generation on ongoing robot actions. Rather than treating inference–execution overlap merely as a systems optimization, StreamWAM couples the two processes: the executed action prefix shapes the predicted visual future, while the model asynchronously infers the next world-action chunk as the robot continues executing the current chunk.
| Asset | Status |
|---|---|
| StreamWAM inference and training code | ✅ Available in this repository |
| Accelerated StreamWAM runtime | ✅ Available in this repository |
| LIBERO and RoboTwin recipes | ✅ Available in this repository |
| FastWAM-Joint-CD checkpoint | ✅ Available on Hugging Face |
| StreamWAM checkpoint | ✅ Available on Hugging Face |
| Technical report | ⏳ Coming soon |
The reference environment uses Python 3.10, PyTorch 2.7.1/cu128, and Triton
3.3.1. pyproject.toml is the canonical dependency definition.
git clone https://github.com/SJTU-DENG-Lab/StreamWAM.git
cd StreamWAM
python -m pip install -U uv
uv syncuv installs PyTorch and torchvision from the official cu128 wheel index. A
compatible NVIDIA driver is required; the host CUDA Toolkit does not need to
match the wheel's bundled CUDA 12.8 runtime exactly.
git clone https://github.com/Lifelong-Robot-Learning/LIBERO.git third_party/LIBERO
uv pip install -e third_party/LIBERO --no-deps
uv run huggingface-cli download Wan-AI/Wan2.2-TI2V-5B \
--local-dir checkpoints/Wan2.2-TI2V-5BLIBERO is supplied as an external source checkout through LIBERO_HOME_PATH.
Its expected source tree contains libero/libero/{benchmark,bddl_files, init_files,assets}. A datasets/ directory is optional for rollout-only use.
Place a compatible StreamWAM checkpoint and its dataset statistics on disk, then run:
PYTHON_BIN=.venv/bin/python \
GPU_IDS=0,1,2,3 \
BACKBONE_PATH="$PWD/checkpoints/Wan2.2-TI2V-5B" \
LIBERO_HOME_PATH="$PWD/third_party/LIBERO" \
CHECKPOINT_PATH=/path/to/ac_stream_checkpoint.pt \
STATS_PATH=/path/to/dataset_stats.json \
bash examples/libero/scripts/launch_streamwam_libero_ac_stream_4gpu.sh \
--ac-stream-acceleratedThe launcher defaults to one trial for every task in libero_spatial,
libero_object, libero_goal, and libero_10. See the
LIBERO guide for checkpoint formats, training,
single-task rollout, and evaluation controls.
To evaluate Stream-WAM across WAM families, we further train FastWAM-Joint with the streaming approach on LIBERO and apply the same design to StarWAM on RoboTwin 2.0 and X-WAM on RoboCasa. All evaluations use four NVIDIA H100 GPUs.
We compare against general purpose robot policies and WAM baselines on task performance, and against WAM baselines on inference efficiency. CD denotes one-step consistency distillation. On LIBERO, we also ablate action conditioning and the slot encoder to assess each component. Best and second best task results are shown in bold and underlined, respectively.
LIBERO evaluation covers four suites: Long, Spatial, Goal, and Object, with 10 tasks per suite and 50 trials per task. We report average success across suites; Episode Time is reported separately for Long and Short tasks in the efficiency results.
| Method | Long | Spatial | Goal | Object | Average ↑ |
|---|---|---|---|---|---|
| OpenVLA | 53.7 | 84.7 | 79.2 | 88.4 | 76.5 |
| π₀ | 85.2 | 96.8 | 95.8 | 98.8 | 94.1 |
| π₀.₅ | 92.4 | 98.8 | 98.0 | 98.2 | 96.9 |
| Motus | 97.6 | 96.8 | 96.6 | 99.8 | 97.7 |
| Fast-WAM | 95.2 | 98.2 | 97.0 | 100.0 | 97.6 |
| FastWAM-Joint-CD | 97.20 | 99.60 | 98.60 | 100.00 | 98.85 |
| FastWAM-RTC | 58.40 | 76.20 | 77.00 | 83.40 | 73.75 |
| Stream-WAM (Ours) | 96.60 | 98.80 | 97.40 | 100.00 | 98.20 |
| Stream-WAM w/o Action Conditioning | 94.40 | 96.40 | 96.60 | 97.60 | 96.25 |
| Stream-WAM w/o Slot Encoder | 95.60 | 98.40 | 96.80 | 99.80 | 97.65 |
RoboTwin 2.0 evaluates 50 tasks with 100 rollout episodes per task. Clean reports the easy setting and Random reports the hard domain-randomization setting.
| Method | Clean ↑ | Random ↑ | Total ↑ |
|---|---|---|---|
| π₀ | 65.92 | 58.40 | 62.2 |
| π₀.₅ | 82.74 | 76.76 | 79.8 |
| Motus | 88.66 | 87.02 | 87.8 |
| Motus from WAN2.2 | 77.56 | 77.00 | 77.3 |
| FastWAM-Joint | 87.8 | 87.32 | 87.56 |
| StarWAM-Joint | 84.8 | 86.0 | 85.4 |
| StarWAM-CD | 79.0 | 79.2 | 79.1 |
| Stream-WAM (Ours) | 87.2 | 88.8 | 87.6 |
RoboCasa follows the standard 24-task protocol, with 50 trials per kitchen manipulation task and average success reported across tasks.
| Method | Average Success ↑ |
|---|---|
| π₀.₅ | 41.4% |
| π₀-FAST | 61.2% |
| π₀ | 62.5% |
| Cosmos Policy | 67.1% |
| X-WAM | 75.42% |
| X-WAM-CD | 75.33% |
| Stream-WAM (Ours) | 75.35% |
We compare standard Joint WAM inference, its distilled 1V10A variant, and Stream-WAM on the same real robot manipulation task using a single NVIDIA GeForce RTX 5090 at a 25 Hz control frequency. The project page provides each rollout from two synchronized camera views.
| Method | Chunk Time | Episode Time |
|---|---|---|
| Joint WAM | 667.1 ms | 68 s |
| Distilled WAM (1V10A) | 402.7 ms | 60 s |
| Stream-WAM (Ours) | 67.8 ms | 33 s |
Task success alone does not characterize runtime efficiency. We therefore report Chunk Time, the latency required to prepare the next action chunk, and Episode Time, the duration of a complete rollout, including inference, execution, and replanning.
| Benchmark | Method | Chunk Time | Episode Time |
|---|---|---|---|
| LIBERO | FastWAM | 493.0 ms | 16.31 s Long / 8.25 s Short |
| LIBERO | FastWAM-Joint-CD | 114.2 ms | 6.89 s Long / 3.74 s Short |
| LIBERO | FastWAM-RTC | 142.3 ms | 6.23 s Long / 3.20 s Short |
| LIBERO | Stream-WAM | 41.0 ms | 5.36 s Long / 3.15 s Short |
| LIBERO | Stream-WAM w/o Action Conditioning | 35.1 ms | 5.20 s Long / 2.92 s Short |
| LIBERO | Stream-WAM w/o Slot Encoder | 36.3 ms | 5.31 s Long / 3.01 s Short |
| RoboTwin 2.0 | StarWAM-Joint | 190.17 ms | 110.22 s |
| RoboTwin 2.0 | StarWAM-CD | 81.21 ms | 102.59 s |
| RoboTwin 2.0 | Stream-WAM | 47.09 ms | 77.48 s |
| RoboCasa | X-WAM | 374.07 ms | 17.36 s |
| RoboCasa | X-WAM-CD | 134.37 ms | 13.04 s |
| RoboCasa | Stream-WAM | 115.98 ms | 9.49 s |
Across all three benchmarks, Stream-WAM reduces both runtime measures while maintaining comparable task success. On LIBERO, it achieves a 12.0× Chunk Time speedup over FastWAM and Episode Time speedups of 3.0× and 2.6× on Long and Short tasks, respectively, with 98.20% average success. On RoboTwin 2.0, relative to StarWAM-Joint, Chunk Time falls from 190.17 ms to 47.09 ms and Episode Time from 110.22 s to 77.48 s, while overall success increases from 85.4 to 87.6. On RoboCasa, relative to X-WAM, Stream-WAM achieves a 3.2× Chunk Time speedup and a 1.8× Episode Time speedup, with comparable average success (75.35% versus 75.42%).
streamwam/
├── backbone/ # Wan2.2 and Cosmos-Predict2 adapters
├── wam/ # MoT, Shared-DiT, and StreamWAM model wrappers
├── modules/ # DiT, ActionDiT, attention, and scheduler modules
├── inference/ # consistency sampling and StreamWAM runtime
├── checkpointing/ # native and FastWAM checkpoint adapters
├── training/ # trainers, losses, and entrypoint
└── data/ # dataset and text-cache utilities
examples/
├── libero/ # LIBERO recipes, rollout, and launchers
└── robotwin/ # RoboTwin recipes and deployment adapters
The arXiv entry is not public yet. For now, please cite the project page:
@misc{denglab2026streamwam,
title = {Stream-WAM: Streaming Your World-Action Model for Real-Time Robot Manipulation},
author = {{DENG Lab}},
year = {2026},
howpublished = {Project page},
organization = {Shanghai Jiao Tong University},
url = {https://sjtu-deng-lab.github.io/StreamWAM/}
}Released under the Apache License 2.0.
StreamWAM builds on ideas and open-source work from FastWAM, StarWAM, X-WAM, StarVLA, DreamZero, LIBERO, Wan2.2, and Cosmos-Predict2.