STEP, Short for Spatiotemporal Training with Explicit Prediction, is a single‑stage learning system that lets a humanoid robot read depth sensors and its own joint sensors together to decide how to move. The method introduces multiple learned queries that fuse depth and proprioception into a 64‑dimensional policy state. During training, the robot’s policy network is updated with Proximal Policy Optimization while a bidirectional masked prediction task makes the robot learn how one sensor modality can predict the other, and an action‑conditioned next‑state predictor trains the policy to anticipate how its actions change the state.

In simulated experiments across fourteen different terrain setups the method achieved a mean success rate of 95.17 %, surpassing two baselines: a version without explicit prediction (91.02 %) and a version without multi‑query aggregation (69.98 %). On real‑world hardware with a 23‑DoF Noetix E1 robot, STEP completed almost all traversals: 98 out of 100 stairs, all 20 platform crossings, 18 of 20 gap passes, and every test on stepping stones and a backwards 15‑cm staircase.

However, the approach is sensitive to visual loss. When depth sensors experienced continuous dropout or a 20 ms delay, success dropped to zero on discontinuous terrains such as gaps and stepping stones. Even on stairs and high platforms, traversal rates fell when fresh visual input was unavailable. These failures point to limits of the current model in handling unreliable or delayed visual data.

Future updates may strengthen the policy’s robustness to visual degradation by improving temporal prediction and handling of missing depth frames.