HOST Open-Sourced: Robots Can Learn New Skills by Watching a Human Video Once

On August 3, Ziy变量机器人 (Ziyou Variable Robotics) released and open-sourced the HOST framework, short for Human-to-robot One-Shot Skill AcquisiTion. It attempts to address a highly practical problem in embodied intelligence: Can a robot learn a task it has never performed before simply by watching a human demonstration lasting tens of seconds—without collecting a new batch of robot data or fine-tuning the model again?

The company’s answer is yes—but it has not yet reached the point where a robot can “learn perfectly at a glance.”

According to officially disclosed data, HOST requires only a single human demonstration video averaging about 29 seconds. Without any model fine-tuning for the new task, it can acquire the skill at inference time, achieving a test success rate of 62% while retaining previously learned capabilities. The company also claims that HOST improves performance by 45% over a zero-shot baseline. Compared with Pi-0.5 plus fine-tuning, it requires 50 times less data and learns skills approximately 500 times faster.

What matters most about these figures is not the 62% itself, but that HOST moves “learning new skills” from the training pipeline into the inference pipeline. If this approach can scale, it could directly transform the cost structure of robot skill deployment.

Instead of Copying Human Movements, It First Predicts the Task Outcome

Traditional approaches to imitation from human videos generally begin by trying to answer one question: What actions did the person perform in the video, and how should the robot reproduce them?

That sounds straightforward, but it is extremely difficult in practice.

Humans and robots do not have the same kind of “body.” Human wrists, fingers, shoulders, and elbows differ from robotic arms and grippers in their degrees of freedom, ranges of motion, and force-control mechanisms. Even when a human and a robot are handling the same cup, directly mapping the human hand’s trajectory into the robot’s joint space may produce a trajectory that is impossible to execute, prone to collisions, or unable to maintain a stable grasp.

This is the long-standing embodiment gap in embodied intelligence: the mismatch between physical forms. The more literally movements are translated, the more they are constrained by differences in body structure.

HOST takes a different route. It does not require the robot to imitate the human’s limb movements frame by frame. Instead, it first predicts: If the task proceeds successfully, what should the environment look like next? The robot then works backward from this expected outcome to determine the actions it needs to execute.

This can be understood as “target-image-driven” behavior: the human demonstration tells the robot what it should see at the next stage, while the robot uses its existing control capabilities to decide how to transform the current scene into the target scene.

For example, suppose a person places an object into a storage bin. HOST does not focus on how many degrees the person’s wrist rotated or exactly when their fingers closed. Instead, it focuses on the sequence of state changes: the object is first picked up, then moved, and finally appears inside the bin. How the robot should move its arm and when it should close its gripper are left to the action policy.

This design is closer to how humans learn through observation. When people watch someone perform an unfamiliar task, they generally do not precisely replicate every muscle movement. Instead, they understand the task stages, expected outcomes, and key constraints, then use their own bodies to complete the task.

A 29-Second Video and Robot Actions Cannot Be Rigidly Aligned by Timestamp

Another easily underestimated challenge in helping robots understand demonstration videos is that humans and robots operate at different speeds.

Suppose a human completes a cutting and preparation step in 10 seconds and then moves on to the next step. At the 10-second mark, the robot may still be working on the previous stage. If the system rigidly aligns the two by timestamp, the robot will be forced to reference an image that is “ahead of schedule,” causing its subsequent action decisions to become misaligned as well.

HOST aligns the sequences by task progress rather than absolute time.

Its process can be summarized in three steps:

1. Encode every frame of the human demonstration video as a vector;
2. Encode every step of the robot’s execution into the same vector space;
3. Use dynamic time warping to nonlinearly match frames from the human video with the robot’s action steps.

Dynamic time warping, or DTW, is not a new algorithm. It has long been used in speech recognition, time-series matching, and similar tasks, where it excels at handling two sequences with different tempos but similar structures. In this context, it effectively allows the human demonstration and robot execution to independently “speed up” or “slow down,” as long as their task stages can still be matched.

As a result, at any given execution step, the robot does not necessarily reference the frame at the same timestamp in the video. Instead, it sees the frame that best matches its current task progress. According to the company, this approach reduces temporal error in task-progress alignment by an order of magnitude.

This part of the technology is not flashy, but it is critical. Many video-imitation systems fail not because the model is completely unable to understand the task, but because their alignment mechanisms are too fragile: if execution falls one step behind early on, the system continues referencing the wrong target afterward. By replacing rigid time synchronization with semantically aligned task progress, HOST effectively adds a dynamic calibration mechanism to long-horizon execution.

Two “Experts”: One Imagines the Future, the Other Takes Action

At the heart of HOST is a cascaded policy model with visual prediction capabilities, built on a dual-expert Mixture-of-Transformers (MoT) architecture. According to the official description, it can be divided into two modules with distinct responsibilities:

• The video expert, or “visual brain”: Processes the human demonstration and images from the robot’s viewpoint, identifies the current task progress, and predicts the visual state that should appear next;
• The action expert, or “motor brain”: Converts the predicted visual outcome into robot actions and controls the robot to complete the operation.

The two do not vote in parallel. Their relationship is more like an upstream-downstream collaboration. The video expert answers, “What should the world look like next?” while the action expert answers, “What should my body do to make the world look that way?”

This also explains why HOST can operate without task-specific fine-tuning. The new demonstration primarily provides the model with information about the task objective and stages, while action execution relies as much as possible on capabilities the robot already acquired during pretraining.

Training is divided into two stages. The first is “same-embodiment pretraining,” in which the robot uses videos of itself performing tasks to learn to predict the post-task state and generate the corresponding actions. This stage establishes the relationship between changes in the robot’s own visual observations and its actions.

The second stage is “human-robot video training,” which focuses on handling cross-embodiment differences between human demonstrations and robot execution. In other words, the model first learns, “What will I see after performing this action?” and then learns, “What do the changes in a human video mean for me?”

This training order makes sense. If the robot has not yet established the relationship between its own actions and environmental outcomes, asking it to imitate human videos directly is likely to reduce cross-embodiment alignment to superficial similarity matching.

The Real Value Lies in “Learning at Inference Time”

The prevailing method for expanding robot skill sets remains, in essence, a project-based workflow: define the task, collect data, clean and annotate it, fine-tune the model, deploy and test it, then collect more data if it fails. Every additional skill requires the development team to repeat the cycle.

This approach can work in structured factories because tasks are fixed, environments are controlled, and production volumes are high enough. Problems emerge in homes, stores, and open warehouses, where tasks are numerous and change rapidly, and many operations may only need to be performed once. Collecting hundreds of robot trajectories specifically to “place an unfamiliar object in a temporary location” is not commercially viable.

HOST matters because it attempts to compress the skill-delivery workflow into:

Show the robot one demonstration, then have it perform the task directly.

If subsequent large-scale validation confirms the approach, developers would no longer need to maintain a separate dataset and fine-tuned model version for every long-tail task. On-site operators might even be able to replace programming with demonstrations. This could affect how robots are deployed more profoundly than simply raising success rates by a few percentage points.

The ability to “retain old skills” is also important. Traditional continual fine-tuning often suffers from catastrophic forgetting: after the model learns a new task, its performance on old tasks declines. By acquiring new skills at inference time, HOST theoretically reduces the disruption to existing capabilities caused by parameter updates. However, more comprehensive experimental results are needed to determine how many skills this “retention” covers and whether it continues to hold after multiple rounds of continual learning.

A 62% Success Rate Shows the Approach Works—but Also That It Is Not Yet Ready for Unsupervised Deployment

The most accurate assessment of HOST is: It demonstrates the feasibility of acquiring robot skills from a single video without fine-tuning, but it has not yet solved the reliability problem.

A 62% success rate is respectable for a research prototype, especially under a single-human-demonstration setting. For a real production system, however, it means that nearly four out of every ten attempts may still fail on average. If the task involves knives, heat sources, glassware, or close human-robot collaboration, that failure rate is clearly unacceptable for direct deployment.

In addition, the officially reported multipliers must be interpreted in light of the complete experimental setup:

• Whether the “45% improvement over the zero-shot baseline” refers to an absolute percentage-point increase or a relative improvement depends on the metric definitions in the paper or code;
• The claim of using 50 times less data depends on how many trajectories the comparison method uses and whether pretraining data is included;
• The approximately 500-fold increase in learning speed likely comes primarily from eliminating task-level fine-tuning, but the costs of video processing, planning, and retries during inference should also be included;
• The generalizability of the 62% success rate depends on which tasks were tested, how many trials were conducted, and which robot platform and camera viewpoints were used.

Some harder problems do not automatically disappear with visual prediction. Examples include occlusion that hides critical steps, difficulty determining the state of deformable objects, the inability to recover contact forces from ordinary RGB video, and human demonstrations that contain actions beyond the capabilities of the robot hardware. For contact-intensive tasks such as tying, tightening, inserting, unplugging, and precision assembly, merely predicting the “future image” may not be sufficient; tactile sensing and force feedback remain irreplaceable.

Open Source Matters More Than a Demo Video

The embodied intelligence field has no shortage of impressive demonstrations. What it lacks are methods that external teams can reproduce, adapt to different robot platforms, and stress-test. HOST’s open-source release may therefore be more valuable than any individual success-case video.

Developers should pay particular attention to four issues:

1. Whether the model weights, training code, and data-processing pipeline are fully available;
2. Whether dynamic time warping and visual representations are sensitive to changes in camera viewpoint;
3. Whether the framework supports different robot arms, grippers, and mobile bases rather than being limited to a specific embodiment;
4. Whether the system can refuse execution when faced with flawed demonstrations, missing steps, or unsafe actions.

If the open-source version can only be reproduced on fixed hardware, from fixed viewpoints, and across a limited set of tasks, HOST is better understood as an inspiring research framework. Only if the community can transfer it to multiple robot embodiments at low cost could it become a foundational capability analogous to “in-context learning” in robotics.

Based on the information released so far, HOST’s most commendable feature is that it does not continue to equate robot learning with simply “collecting more trajectories and training larger models.” Instead, it reframes the problem: How can a robot use its existing priors to extract task structure from a single observation and invoke its existing action capabilities to complete the task?

This route may not replace robot-data training. A more realistic combination may be for large-scale pretraining to provide general visual, action, and control capabilities, while inference-time learning frameworks such as HOST handle long-tail tasks at deployment sites. The former determines the lower bound of a robot’s capabilities; the latter determines how quickly it can adapt to new requirements.

Robots that can learn after watching once are still far from human-level capability. But the shift from fine-tuning separately for every skill to using only a video lasting tens of seconds points toward a significant reduction in the unit cost of skill acquisition.

References

• ITHome: Ziyou Variable Robotics Releases and Open-Sources the HOST Framework, Enabling Robots to Learn New Skills from Video — Covers HOST’s core mechanisms, dual-expert architecture, and officially reported experimental data.
• Zhihu: HOST Learns Entirely New Manipulation Tasks from a Single Human Demonstration Video — Provides additional details on average demonstration length, skill acquisition without fine-tuning, and performance comparisons.