30,000 Hours of Data Give Robots a Sense of Touch

Robots Are Finally Using More Than Just Their Eyes

Xinzhi Embodied Intelligence and Fudan University recently released a series of research成果 related to tactile intelligence, open-sourcing the projects, data, and models together. The released dataset contains more than 30,000 hours of visual-tactile interaction video, approximately 1.4 million manipulation clips, and 3.3 billion time steps. More importantly, the team is attempting to convert tactile signals generated by different sensors and robot embodiments into a unified representation that models can learn jointly.

The significance of this work goes beyond the addition of yet another large-scale dataset.

Over the past two years, most progress in embodied intelligence has been built on vision-language-action models, or VLAs: cameras observe the environment, large models interpret the task, and action modules control robotic arms to execute it. But once robots truly enter factories, warehouses, and homes, the hardest problems are often not about “seeing where the cup is,” but about the final few millimeters after contact occurs: Has the cup started to slip? Has the cardboard box been crushed out of shape? Has the plug already pressed against the socket? Will the gripper apply enough force to damage the component?

It is difficult to obtain this information reliably using cameras alone.

30,000 Hours First Addresses the Data-Scale Problem

The scale of this dataset deserves attention in its own right.

More than 30,000 hours, 1.4 million manipulation clips, and 3.3 billion time steps mean that the dataset does not merely record whether a grasp succeeded. It covers the continuous states before contact, during contact, and after an action is adjusted. For tactile models, continuous change is usually more important than the result in a single frame: one pressure map can only tell the model what the object looks like under pressure at that moment, while a time series allows it to determine whether the object is slipping, whether the applied forces are becoming unbalanced, or whether an insertion action has encountered resistance.

This can be compared to autonomous-driving data. A single road image can help a model recognize vehicles and pedestrians, but braking, avoidance, and trajectory prediction depend on continuous video and vehicle-state data. The same is true of touch: a robot needs to know how pressure changes with its actions, rather than merely memorizing a particular instant of contact.

Based on the publicly available information, this dataset has at least three direct applications:

• Training joint visual-tactile representations so that models can correlate changes in images with contact feedback;
• Learning physical states such as slipping, collisions, compression, and deformation that are difficult to infer accurately from vision;
• Providing high-frequency feedback to action policies so that robotic arms can correct themselves in real time after contact, rather than fully executing a pre-generated trajectory.

Of course, data scale does not automatically translate into model capability. Tactile data is particularly prone to having “many hours but little useful information”: a robotic arm may remain out of contact for extended periods, and adjacent time steps may be highly repetitive. What truly determines the value of the data for training is whether the manipulation tasks are sufficiently diverse, whether the sensors have been calibrated, and whether the dataset preserves the relationships among force, pose, action commands, and contact states.

Thirty thousand hours is therefore an important starting point, but it cannot be compared with other datasets solely by video duration. The proportion of valid contact segments, task distribution, sensor types, and benchmark results will be more important to watch going forward.

Tactile “Dialects” Are an Even Tougher Problem Than Data Scarcity

Another focus of this work is unifying the “dialects” of tactile data.

One important reason visual models have scaled so rapidly is that data from different cameras comes in relatively standardized forms: most of it can ultimately be converted into RGB images or video frames. Tactile sensors, however, are far less standardized.

Some sensors output pressure matrices, while others generate images from the deformation of flexible surfaces. Some record three-axis forces and torques, while others can also estimate tangential force, contact area, slip direction, and material hardness. Different devices have different sampling rates, spatial resolutions, mounting positions, coordinate systems, and measurement ranges. Even when two sensors both output a “tactile image,” the physical meaning represented by each pixel may be entirely different.

As a result, data collected by one company can usually be used only with its own hardware and models. Change the gripper or sensor, and the training pipeline must be adapted all over again. Although the industry appears to have accumulated a substantial number of tactile samples, they are actually scattered across mutually incompatible data silos.

What Xinzhi Embodied Intelligence and Fudan University are trying to solve is precisely how to enable models to understand these signals across devices. This is not simply a matter of combining different datasets. It requires establishing a representation that acts like an “intermediate language”: the underlying sensors continue to speak their own dialects, while the model sees contact states that are as consistent as possible.

If this approach proves effective, its significance could exceed any ranking improvement achieved by a single model. The scaling of large vision models depends on standardized data formats and training interfaces; tactile intelligence needs similar infrastructure. Otherwise, every robotics company must repeatedly collect, calibrate, and clean its own data, making it difficult to establish a general-purpose pretraining paradigm for tactile models.

Why Adding Touch to a Model Can Actually Make It Worse

Tactile intelligence does not become effective simply by connecting another sensor and adding a few more input channels to a model.

Recent embodied-intelligence research has already revealed a counterintuitive phenomenon: directly adding tactile signals to an existing model can reduce rather than improve task success rates. The reason is not mysterious—vision, language, and touch operate on different timescales.

Language describes goals that unfold over seconds or even minutes, such as “pick up the earphones and place them in the packaging box.” Vision is typically updated at dozens of frames per second to identify positions and poses. Tactile feedback may need to be processed at an even higher frequency because slipping and collisions happen extremely quickly. If a model crudely compresses all three modalities into the same sequence and generates actions at the same cadence, tactile signals can easily become noise.

A well-designed system is more like hierarchical control:

1. A high-level model understands the task and decides whether the next step is to grasp, insert, or rotate;
2. A mid-level policy plans the robotic arm’s trajectory and selects its approach direction and target pose;
3. A low-level controller continuously adjusts the action based on changes in pressure, slipping, and force.

When humans pick up a paper cup, they do not stop every few seconds to reconsider their grip strength. The brain provides the intention to “pick it up,” while the fingers and nervous system rapidly close the feedback loop after contact. What tactile models truly need to add is this high-frequency, low-latency feedback loop that existing robotic systems lack.

This also means that scaling in the style of large language models is not the only answer for tactile intelligence. In addition to more data, models must handle synchronization, latency, control frequency, and safety boundaries. A model that scores highly in offline evaluations but has an inference latency of several hundred milliseconds will struggle to perform low-level corrections in precision assembly tasks.

Open Source Matters More Than “First” or “Largest”

The pace of tactile-model releases is clearly accelerating. In July 2026 alone, Chinese teams released native visual-tactile datasets containing thousands of hours of data, along with VTLA embodied models. Other companies announced that they had accumulated even larger volumes of manipulation data incorporating tactile signals. Each organization is emphasizing data scale, hardware coverage, and cross-task generalization.

Compared with claims of being the “first” or “largest,” what is more commendable about this release from Xinzhi Embodied Intelligence and Fudan University is that the project, data, and models have all been open-sourced together.

If only a paper is released, external teams have difficulty determining whether the model’s improvements come from the data, sensors, or training techniques. If only the model is released, researchers cannot investigate cross-hardware transfer or data cleaning. If only a small amount of demonstration data is provided, it is insufficient to verify whether pretraining can scale. Releasing all three at least gives developers the opportunity to reproduce the results, fine-tune the model, and redesign the representation.

For robotics companies, open-source data can also reduce cold-start costs. Teams can evaluate whether a tactile encoder is worth integrating into an existing VLA without first building an entire data-collection pipeline, or test whether data from their own grippers can be mapped into the unified representation. Even if they ultimately still need to collect proprietary data for specific scenarios, pretrained models may reduce the number of samples required compared with starting from scratch.

However, after open-sourcing the work, the project team still needs to address several key questions:

• Does the data license permit commercial training and distribution of derivative models?
• Will the entire 30,000-hour dataset be released, or will it be published in batches or only as processed features?
• Is the data evenly distributed across different hardware platforms, or does a single sensor account for most of the samples?
• Can the model transfer to previously unseen grippers, robotic arms, and tactile sensors?
• Compared with vision-only policies, how much does adding touch improve performance in tasks involving slipping, flexible objects, and precision insertion?
• Do the inference frequency and deployment compute requirements support closed-loop control on real robots?

If this information remains unavailable, the 30,000 hours may amount to little more than an eye-catching scale metric. If the project can deliver a standardized toolchain, cross-embodiment benchmarks, and reproducible results, it has the potential to become a general-purpose data foundation for tactile intelligence.

Not Every Robotic Task Will Benefit

Touch will not replace vision, nor is every embodied task worth equipping with tactile sensing.

For tasks such as moving cardboard boxes through open spaces or transporting materials along fixed routes, visual positioning and conventional force control are usually sufficient. Adding high-resolution tactile sensors introduces costs related to hardware, durability, cable routing, calibration, and real-time computation. If the task has a high tolerance for error, the return on investment may not be worthwhile.

Tactile sensing is better suited to scenarios where success or failure is determined by the “final centimeter”:

• Precision assembly of connectors, earphones, and small electronic components;
• Manipulation of flexible or easily deformable objects such as plastic bags, cardboard boxes, and clothing;
• Grasping fragile objects such as glassware and fruit;
• Tasks with severe occlusion where cameras cannot directly observe the contact surface;
• Industrial processes that require the detection of slipping, jamming, missed grasps, and abnormal collisions.

What these tasks have in common is that the robot must not only complete an action but also recognize when it has made a mistake, then stop or correct itself before the error escalates. On real production lines, this capability is often more valuable than completing a complex action once in a demo.

Embodied Models Are Starting to Make Up for Missing Infrastructure

In the past, the embodied-intelligence industry invested substantial resources in visual data, language instructions, and action trajectories, hoping to replicate the path by which large models develop emergent capabilities through massive pretraining. But the physical world is different from internet text: every contact a robot makes is affected by friction, materials, assembly tolerances, and sensor latency, while many states never appear fully in images at all.

The 30,000 hours of open-source visual-tactile data from Xinzhi Embodied Intelligence and Fudan University at least provide the industry with a previously scarce public training resource. It may not immediately produce a general-purpose robotic “brain,” but it could advance more practical work: standardizing data formats, establishing cross-sensor evaluations, validating tactile pretraining, and truly connecting high-level VLAs with low-level feedback control.

Our assessment is that tactile intelligence will not rapidly consolidate the market through parameter scale alone, as language models have. Instead, it is more likely to become a foundational capability layer within embodied systems. Future robot models may not consist of a single network that handles every decision, but rather a system combining high-level reasoning, motion planning, and tactile closed-loop control.

Thirty thousand hours cannot give robots a complete “sense of touch,” but it creates the first real opportunity to turn tactile intelligence from a collection of proprietary techniques confined to individual laboratories into a public problem that developers can jointly validate and improve.

References

• Zhihu: After Robots Learn to “See” and “Speak,” How Can We Give Them a True “Sense of Touch”? — Explains the role of tactile data in embodied manipulation and reviews related datasets and industry developments in China.