HITCH (Harness Interfaces To Command Humanoids) is a set of tools that translate a vision‑language model’s (VLM) language instructions into actions for a humanoid robot. The harness supplies two key interfaces. First, perceptual calls return the 3D position of any point the VLM highlights in its head‑camera image, eliminating the need to guess distances. Second, execution calls give the VLM commands for pelvis height, torso pitch and yaw, hand waypoints, footstep placement, and gaze. A planner converts these body‑part commands into whole‑body reference trajectories, and a learned tracker called HEFT executes them on a Unitree G1 robot. The tracker is further trained on the planner’s references collected while teachers solve five base tasks, reducing hand error to 0.6 cm and walk error to 5 % on the SIMPLE benchmark.

A fine‑tuned Qwen3.8‑27B VLM, trained on teacher trajectories, observes one perception or execution request per turn and conditions on the full history of observations and actions. In zero‑shot tests, the model solves 51 % of SIMPLE episodes and 42 % of HumanoidArena episodes. Adding just five task‑specific trajectories per SIMPLE task raises SIMPLE success to 95 %. The model can also perform a carrying task on a real G1 robot with no task‑specific training, succeeding in 13 of 20 carrying episodes and 6 of 10 navigation episodes.

The results show that both accurate scene measurement and precise command execution are essential: removing the perception interface lowers SIMPLE success from 51 % to 2.8 %. However, the planner currently cannot generate sitting or obstacle‑over‑steps, limiting the robot’s motion repertoire. Adding SIMPLE trajectories also reduces HumanoidArena success from 42 % to 28 %, indicating that the agent over‑fits to repeated patterns. Real‑world tests are further constrained by the robot’s rubber hands, which cannot perform finger‑level manipulation.