X SQUARE ROBOT has released and open-sourced HOST, short for Human-to-robot One-Shot Skill Acquisition. The framework enables a robot to acquire a new manipulation skill from a single human demonstration video, without task-specific fine-tuning or parameter updates.
HOST achieves an average success rate of 62% after watching a human video lasting just 29 seconds on average. Compared with supervised fine-tuning using 50 robot demonstrations per task, it uses 50 times fewer demonstrations and acquires each skill 507 times faster.
Unlike conventional imitation methods that try to map human movements directly to robot actions, HOST first predicts what the demonstrated outcome should look like from the robot’s perspective, then derives the actions needed to achieve it.
Learning by Following Task Progress
HOST is inspired by observational learning in humans. When people watch someone perform an unfamiliar task, they can anticipate the intended result and adapt the observed procedure to their own bodies without extensive practice.
The first challenge for a robot is determining which part of a human video corresponds to its current stage of execution.
Human demonstrations and robot actions rarely proceed at the same speed. After ten seconds, for example, the person in a video may have completed one step while the robot is still working on it. Aligning the two sequences by clock time would therefore provide the robot with guidance from the wrong stage.
HOST instead maps the human video and robot trajectory onto a shared task-progress manifold. It then establishes frame-level correspondences according to task progress rather than elapsed time.
In the paper’s evaluation, this approach reduced the mean absolute progress-alignment error from 0.079 with timestamp-based matching to 0.006—an improvement of more than one order of magnitude.
Predicting Outcomes Before Actions
The second challenge is embodiment mismatch. A robot’s physical structure, viewpoint and workspace differ substantially from those of a human, making direct motion imitation unreliable.
HOST addresses this problem through a three-stage cascade called self-grounded prediction.
First, it localizes the robot’s current progress within the human demonstration. Next, it translates the upcoming part of the demonstration into predicted future observations from the robot’s own perspective. Finally, it derives the robot actions required to reach those predicted states.
For example, after watching a person pick up a cup, HOST does not attempt to copy the person’s arm movements. It predicts what the scene should look like after the robot has picked up the cup, then generates the actions needed to produce that result.
The framework uses a dual-expert Mixture-of-Transformers architecture. A video expert handles task localization and future visual prediction, while an action expert converts the predicted observations into robot commands.
A Two-Stage Training Strategy
HOST is trained in two stages.
During same-embodiment pretraining, it learns from pairs of robot trajectories that perform the same task. This builds its ability to follow a demonstrated procedure, predict future robot observations and generate corresponding actions.
During human-to-robot adaptation, the model learns from paired human videos and robot trajectories, extending these capabilities to human demonstrations.
The first stage uses 193,462 robot trajectories spanning 229 tasks. The second uses 5,847 human–robot pairs. This design reduces reliance on scarce paired data showing humans and robots performing the same behavior.
Once training is complete, HOST’s parameters remain frozen. New skills are acquired entirely at inference time from a single video.
62% Success with 50× Less Data
The researchers evaluated HOST across 50 previously unseen manipulation tasks involving different objects, tools and manipulation primitives. The paper reports that HOST acquired an executable skill for every task from a single human video.
Quantitative comparisons with other methods were conducted on a representative subset of eight new tasks, with 20 trials per task.
HOST achieved an average success rate of 62%. AWDA, the strongest one-shot visual imitation baseline, achieved 19%, giving HOST a 43-percentage-point advantage. HOST also outperformed the strongest language-conditioned zero-shot baseline by 45 percentage points.
The team then compared HOST with imitation-learning models fine-tuned on 10, 20 and 50 teleoperated robot demonstrations per task. At 50 demonstrations, the strongest fine-tuned baseline, Wall-OSS + SFT, reached 56%—still below HOST’s 62% from a single human video.
Collecting 50 robot demonstrations and fine-tuning the models took approximately 4.0 to 4.9 hours per task. HOST required only the time needed to record one human video: 29 seconds on average. It was therefore 507 times faster than the fastest fine-tuning baseline, π0.5 + SFT.
Retaining Previously Mastered Skills
Because HOST does not update its parameters when acquiring a new skill, it avoids the catastrophic forgetting caused by task-specific fine-tuning.
Across seven previously mastered tasks, HOST retained 99% of its original performance. After fine-tuning on 50 demonstrations, Wall-OSS + SFT retained 43%, HOST-base + SFT retained 22%, and π0.5 + SFT retained 20%.
A human demonstration can also be stored and retrieved when the same task returns. In this sense, each video functions like a reusable recipe, allowing the robot to maintain a growing library of acquired skills without repeated training.
Robust Under Real-World Changes
HOST was further tested under four deployment perturbations. Its average success rate was:
61% under lighting changes
58% with out-of-distribution objects
56% after scene replacement
53% when a person moved an object during execution
The default success rate was 62%.
Even when an object was displaced midway through a task, HOST could reassess its progress, correct its actions and continue. This suggests that the framework is not simply replaying a memorized motion sequence, but dynamically following the demonstrated procedure.
Making Robot Teaching More Accessible
HOST shifts robot skill acquisition from a training-time fine-tuning loop to an inference-time process. Instead of relying on specialists to collect robot demonstrations and retrain a model for every new task, an ordinary user can teach the robot through a short human video.
The research paper, code and supporting materials are now publicly available.
Resources
https://host-site.host-robotics.workers.dev
https://arxiv.org/abs/2607.20033
https://github.com/CGuangyan-BIT/HOST
https://huggingface.co/papers/2607.20033