The new model strengthens action-conditioned prediction, supports stable 60-second autoregressive rollouts, and achieves a 0.93 correlation between simulated and real-world policy success rates.
SHENZHEN, China — X Square Robot today introduced WALL-SS, a next-scale autoregressive world model designed to make robot simulation more controllable, stable over long horizons, and representative of real-world outcomes.
World models allow robots to preview the consequences of candidate actions before executing them in the physical world. By providing a learned “virtual training ground,” they can support robot planning, policy evaluation, and data generation while reducing the cost and risk of real-world trial and error.
Existing world models, however, continue to face three major challenges. First, they may generate visually plausible futures while paying insufficient attention to the specified robot actions. Second, errors can accumulate during autoregressive rollout, causing objects, robot states, and scene geometry to drift or collapse. Third, success inside a generated environment does not necessarily translate into success on a physical robot, limiting the value of world models as policy-evaluation tools.
WALL-SS addresses these challenges through a next-scale autoregressive architecture. Instead of generating an entire video segment in a single diffusion process, the model constructs each future clip as a sequence of coarse-to-fine predictions. Broad scene structure and motion are established first, followed by progressively finer visual and physical details.
In the accompanying research paper, the system is presented under the research name WALL-ARWM, abbreviated as AR-WM.
Making Robot Actions Matter
In contact-rich manipulation, a small difference in an end-effector trajectory or gripper command can separate a successful grasp from a collision or failed pickup. Yet these outcomes may look nearly identical before contact, making it easy for a video model to follow visual priors instead of the supplied controls.
WALL-SS introduces scale-aligned action conditioning, which makes the robot’s controls directly available throughout the coarse-to-fine generation process. Action information is not injected only once as a high-level prompt. Instead, it is aligned with the relevant video clip, camera view, and visual scale at each stage of prediction.
The model converts robot commands into view-specific visual conditions using camera calibration and the planned kinematic trajectory. These conditions include projected end-effector poses, local coordinate axes, gripper states, and projection-validity information. Synchronized information exchange across cameras further helps ensure that different views describe the same underlying physical transition.
This design allows WALL-SS to generate counterfactual futures from the same initial scene. When given leftward, forward, or rightward control trajectories, for example, the generated robot follows correspondingly different paths rather than collapsing to a single scene-driven outcome.
On the paper’s embodied video-generation benchmark, WALL-SS achieved an Action Following score of 0.29, the highest reported result among the evaluated models. It also achieved a Trajectory Accuracy score of 0.539, compared with 0.251 for its InfinityStar backbone—an improvement of approximately 115%.
Together, these results indicate that the model is not merely reacting to the presence of an action signal. It is more accurately reproducing the direction, timing, magnitude, and visual consequences of the prescribed robot motion.
Preserving Coherence Over Longer Rollouts
Autoregressive world models generate new visual states from their own previous predictions. Even small errors can therefore become part of the context for the next prediction, gradually producing object drift, identity changes, discontinuities, or unrealistic interactions.
Keeping every historical state at full resolution is not a practical solution, because memory and attention costs would continue to grow with rollout length. Retaining only the most recent clips, on the other hand, can cause the model to forget persistent scene and task states.
WALL-SS addresses this trade-off with time-scale memory. Recent history is retained at finer representational scales to preserve motion, pose, and contact information, while older history is kept at progressively coarser scales to preserve longer-term information such as scene layout and object identity. A compact representation of the initial observation remains available as a persistent anchor.
Each retained visual state also remains paired with the action information from the same physical interval. This helps the model preserve the relationship between an action and its consequences even as older visual history is represented at lower precision.
Under a fixed retention schedule, the number of resident memory records and the model’s peak working-memory requirement remain bounded rather than growing with the number of completed clips.
WALL-SS further introduces scale-wise dream forcing to reduce the gap between training and inference. During training, the model is exposed to corrupted historical states and contexts produced by a lagged version of the generator. It therefore learns to continue predicting from imperfect, self-generated history instead of relying exclusively on clean ground-truth context.
The paper evaluates autoregressive rollouts extending to 60 seconds. Across successive temporal windows, the model is assessed on action-grounded trajectory error, persistent-state consistency, cross-clip continuity, perceptual quality, and multi-camera consistency. Qualitative comparisons at 5, 30, and 60 seconds show that WALL-SS better preserves scene geometry, object identity, and coherent task progression than the evaluated baselines.
Connecting Virtual Evaluation With Physical Results
A useful world model must do more than generate realistic video. It must preserve the success and failure patterns that a robot policy would encounter in the physical world.
WALL-SS applies on-policy alignment to the visual generator. With robot actions held fixed, the model generates fresh visual rollouts and optimizes them using two complementary reward signals. One measures whether the generated transition follows the specified action; the other measures long-term consistency across multiple clips, including boundary continuity, object persistence, contact evolution, and agreement across camera views.
This alignment updates the visual world model—not the robot controller. Replay of real trajectories and regularization against a reference model are used to preserve the pretrained visual distribution during optimization.
For closed-loop evaluation, researchers ran the same frozen external robot policy both inside WALL-SS and on a physical robot. The experiment covered:
Six manipulation tasks
Five policy checkpoints representing different levels of competence
Twenty matched initial configurations per task
A total of 600 paired simulated and real-world rollouts
Across the 30 task-and-checkpoint success-rate cells, generated and physical results achieved a Pearson correlation of 0.93, with a mean absolute success-rate error of 0.062.
WALL-SS also preserved 89% of the non-tied within-task checkpoint orderings, meaning it generally identified which policy checkpoint performed better on the same task. At the individual-episode level, simulated outcomes predicted matched real-world success or failure with a balanced accuracy of 0.88.
These results suggest that WALL-SS can serve as a useful policy-evaluation environment—not simply by producing plausible images, but by approximating how differences in policy quality affect physical task outcomes.
A Lower-Cost Path to Robot Capability Development
Real-world robot training and evaluation are expensive, time-consuming, and potentially risky. A reliable world model can allow developers to test candidate behaviors, compare policy checkpoints, and identify likely failures before committing extensive time to physical experiments.
By combining action-grounded next-scale generation, bounded long-horizon memory, robustness to self-generated context, and reward-based visual-dynamics alignment, WALL-SS moves toward a more dependable virtual environment for embodied AI research.
The goal is to shorten the cycle between policy development and physical deployment: robots can first explore and validate behaviors in a generated environment, while the most promising policies proceed to testing in homes, factories, logistics facilities, and care settings.
Project page: https://x2robot.com/pages/ss
Research paper: https://github.com/X-Square-Robot/wall-ss/blob/main/wall-ss-paper.pdf
GitHub: https://github.com/X-Square-Robot/wall-ss