Hesai Kosmo: Replicate Reality in 15 Minutes

Hesai Wants to Build More Than Just “Eyes” for Robots

On October 14, Hesai Technology officially unveiled the Kosmo One AI Spatial Camera and its companion cloud platform, Kosmo World. The former captures real-world spaces, while the latter handles 3D model generation, scene editing, semantic segmentation, and intelligent completion—ultimately transforming physical environments such as shopping malls, hotels, and factories into training data that robots can use.

Hesai’s headline specifications are straightforward: Kosmo One features a one-inch CMOS sensor, supports resolutions of up to 8K and 360° spatial coverage, and can reconstruct a 100-square-meter real-world space in 3D in as little as 15 minutes.

It is important to note that “8K” refers to the image-capture specification and does not equate to the geometric accuracy of the final 3D model. Likewise, “reconstructing 100 square meters in 15 minutes” is better understood as a benchmark for a typical scenario. Actual speed will also be affected by spatial complexity, occlusion, lighting, moving pedestrians, and the cloud-processing pipeline.

Even without discussing the limits of its accuracy, however, this product suite addresses one of the most practical bottlenecks in embodied AI today: the industry has no shortage of robot demonstration videos. What it lacks is data that can be produced at scale, incorporates real geometry and spatial semantics, and can be fed directly into training workflows.

In other words, Kosmo One may look like a camera, but what Hesai really wants to sell is a “real-world data production line.”

From Capturing Images to Actual Training, There Is an Entire Data Pipeline in Between

Conventional cameras can record color, while lidar can capture depth and geometric structure. But robot training requires far more than a video clip or a point cloud.

When a service robot enters a shopping mall, it needs to know where the walls and glass doors are, whether it can pass between shelves, whether tables and chairs are movable objects, and how ramps differ from steps. For robotic arms, the requirements become even more granular, extending to object contours, surface materials, graspable regions, and collision relationships.

Kosmo’s approach is to integrate Hesai’s existing strengths in lidar, high-definition imaging systems, and 3DGS reconstruction technology into a single pipeline:

1. Capture real-world spaces: Simultaneously acquire textures, colors, lighting, shadows, and 3D geometry;
2. Reconstruct the space: Convert discrete images and point clouds into navigable, interactive 3D scenes;
3. Understand scene content: Use semantic segmentation to identify walls, floors, furniture, passageways, and other elements;
4. Fill gaps in captured data: Use intelligent completion to address occlusion, missing regions, and incomplete textures;
5. Output training assets: Organize scenes into Sim-Ready data suitable for pre-training, post-training, and evaluation.

Here, 3DGS stands for 3D Gaussian Splatting. It represents a space using large numbers of 3D Gaussian primitives with position, color, opacity, and shape parameters. It generally outperforms earlier NeRF approaches in novel-view rendering speed and visual quality, while also being better suited to real-time navigation, content editing, and interactive presentation.

However, “looking realistic” does not mean that a scene can be used directly to train robots.

Film, television, and tourism applications are primarily concerned with visual quality, whereas robot simulation places greater emphasis on dimensions, coordinate systems, collision meshes, surface properties, object classes, and interactive states. If a glass door looks convincing in an image but lacks accurate collision boundaries and opening-and-closing behavior, it still has limited value for navigation or manipulation training.

That is why Kosmo World deserves more attention than Kosmo One. The hardware addresses how to capture data quickly, while the cloud platform determines whether that data can be transformed from an “attractive 3D scan” into a truly computable, editable, and reusable training asset.

Why Robotics Companies Increasingly Need Real 3D Spaces

The data problem in embodied AI is not quite the same as it is for large language models.

Large models can learn extensive statistical patterns from online text and images. Robots, by contrast, must contend with scale, depth, friction, collisions, and occlusion. A 2D video may tell a model what a table looks like, but it may not accurately indicate whether a robot can pass between its legs, how far a cup is from the edge, or whether a reflection in glass represents an obstacle or merely a lighting effect.

Robot training data currently comes from roughly three sources:

• Human teleoperation and on-site data collection: Offers the highest realism, but carries high equipment, labor, and time costs;
• Fully synthetic simulation environments: Fast to generate and inherently fully annotated, but prone to domain gaps with the real world;
• Internet videos and 2D images: Available at the greatest scale, but lacking stable geometry, depth, and interaction information.

Kosmo attempts to establish a fourth path: rapidly scan a real-world space, then organize it into a simulation environment that can be repeatedly edited and reused.

This effectively turns “going on-site to train once” into “copying the site into the training set.” A robotics team could move shelves, add obstacles, or alter aisle layouts within the same shopping-mall model, creating numerous long-tail scenarios without having to close off the physical site and test there each time.

This type of data can support three stages:

Pre-Training: Supplementing Spatial Common Sense

Real 3D scenes can provide vision-language models and world models with more stable depth, scale, and spatial relationships. This allows models not only to recognize “what an object is,” but also to understand “where it is and how it relates to other objects.”

Post-Training: Practicing Specific Tasks

Robots can practice navigation, obstacle avoidance, path planning, and object manipulation in reconstructed hotels, restaurants, stores, or factories. Compared with entirely procedurally generated scenes, real-world scans often preserve more irregular structures and real-world noise.

Evaluation: Reproducing Problems and Creating Long-Tail Cases

Teams can digitize real-world sites where failures have occurred and repeatedly test different algorithm versions. They can also modify lighting, obstacles, and furniture layouts to assess whether a model remains stable as its environment changes.

This may be more valuable than simply increasing model parameters. For robots, parameter count determines the upper bound of capability, but the coverage of the training environment determines whether they fail immediately after leaving the laboratory.

Compared with Conventional 3D Scanning, Kosmo’s Advantage Is Workflow Compression

Digitizing real-world spaces is not a new requirement. Photogrammetry, professional laser scanners, panoramic cameras, and NeRF tools can all perform similar tasks, but these solutions generally address only part of the problem.

Photogrammetry requires large numbers of overlapping photographs and places high demands on capture paths, lighting, and post-processing. Professional laser scanning offers high accuracy, but the equipment is often expensive, the workflow complex, and specialists are typically required to process the point clouds. Panoramic cameras are easy to deploy but capture limited geometric information. Purely vision-based NeRF or 3DGS solutions can deliver strong visual results, but geometric reliability remains challenging when dealing with low-texture walls, glass, reflective surfaces, and complex occlusion.

Kosmo One’s competitiveness may not come from being the first to introduce any one technology. Rather, it lies in compressing capture, reconstruction, semantic processing, and asset management into a relatively standardized workflow.

If its claim of reconstructing 100 square meters in 15 minutes can be consistently replicated in ordinary commercial venues, complex factories, and dynamic public spaces, the resulting change will not simply be that “scanning is several times faster.” Instead, 3D capture could shift from a one-off engineering project into an everyday operation.

The transition would resemble digital cameras replacing film: the benefit was not merely faster shutter operation, but the elimination of the entire intermediate workflow of developing, scanning, and converting files. Kosmo similarly aims to transform spatial modeling from a project-based service delivered by specialist teams into a standardized task that store employees or ordinary operations staff can perform.

Partnering with Amap, Starting with 1,000 Real-World Sites

Hesai also announced a partnership with Amap to launch an initial batch of 1,000 pilot sites spanning shopping malls, retail outlets, restaurants, hotels, and branded stores. The goal is to fully scan and reconstruct store interiors as high-precision 3D spaces rather than stopping at conventional 2D floor plans.

This partnership offers value on two levels.

First, indoor spaces remain a relatively weak area for existing mapping capabilities. A 2D map can guide users to a shopping mall, but it struggles to fully represent the mall’s internal 3D structure, store layouts, and navigational relationships. If continuously updated, 3D reconstructions could support indoor navigation, spatial search, in-store guidance, and digital operations.

Second, these 1,000 pilot sites will also serve as a stress test for Kosmo’s data-production capabilities. Scanning a tidy room in a laboratory is not difficult. The challenge is to consistently obtain usable data in commercial spaces with complex lighting, dense foot traffic, and frequently changing displays.

Amap provides access to real-world scenarios and spaces, while Hesai supplies the capture and reconstruction tools. Compared with creating a single visually impressive showcase, this kind of large-scale deployment will be more effective in determining whether Kosmo is merely a demonstration device or data infrastructure that can be deployed at scale.

Of course, 1,000 pilot sites do not automatically equal 1,000 datasets that can be used directly for robot training. Scene authorization, personal privacy, merchant information, data anonymization, update frequency, and permitted uses must all be clearly defined. Particularly in hotels, restaurants, and retail environments, scanning devices can easily capture faces, screens, price tags, and even internal operational information. Data governance cannot be deferred until after the product reaches scale.

Hesai’s Strategy: Moving from Sensor Supplier into the Data Layer

Hesai has historically been known primarily for lidar. The significance of Kosmo is that the company is no longer content to supply only sensors for robots or vehicles; it is beginning to move into the processing, management, and training stages of perception data.

This progression is not unexpected. Lidar allows machines to “see” geometric structures. Kosmo turns those structures and images into persistent spatial assets, and the cloud platform then processes those assets into training data. Combined with Hesai’s newly expanded robotics power-module business, the company’s footprint is gradually extending from perception to understanding and execution.

However, this also means Hesai will be entering a market that is more complex than hardware.

Cameras and lidar can be compared relatively clearly in terms of resolution, range, point rate, and cost. A 3D data platform must answer many more questions:

• What are the absolute dimensional error and long-term consistency of the reconstruction results?
• Which classes does semantic segmentation support, and can customers define their own?
• How are dynamic objects, transparent materials, reflective surfaces, and low-texture regions handled?
• Can data be exported to mainstream robot simulators and digital-content tools?
• Do scenes include reliable collision meshes, navigation meshes, and physical properties?
• Who owns the raw data, model assets, and derived training data, respectively?
• How much do cloud reconstruction and large-scale storage cost?

These metrics are harder to communicate than “8K,” but they will directly determine whether developers are willing to integrate Kosmo into their training pipelines.

What Kosmo Really Needs to Prove Is Not That It Can Scan, but That Its Outputs Can Be Reused at Scale

From a product-strategy perspective, Kosmo is addressing a real problem, and the timing is appropriate.

The robotics industry is shifting from demonstrating isolated tasks to pursuing generalization across environments. Continuing to rely on engineers to manually build a small number of simulation scenes makes it difficult to cover the enormous variety of real-world layouts and long-tail conditions. Whoever can replicate real-world spaces at lower cost has an opportunity to become a data supplier for the robotics era.

However, several key questions about Kosmo remain unanswered publicly.

The first is accuracy. Hesai has disclosed resolution and reconstruction speed, but developers are more concerned with centimeter- or even millimeter-level errors, localization drift across rooms, and completeness when dealing with complex materials.

The second is formats and ecosystem compatibility. Robotics companies already use different simulators, asset formats, and training frameworks. If Kosmo World can only display scenes within its proprietary platform, its value will be closer to that of a 3D content tool. It will truly qualify as physical AI infrastructure only if it can integrate seamlessly into existing simulation and training pipelines.

The third is cost. Rapid capture does not necessarily mean low cost. Equipment prices, cloud computing, storage, model updates, and manual review must all be taken into account. For customers that need to scan tens of thousands of stores, the cost per square meter and the cost of updates matter more than the speed of a single demonstration.

Finally, there is the question of training benefits. High-fidelity scenes do not inherently produce better robot models. Hesai still needs to demonstrate—through public benchmarks or customer results—how much Kosmo data actually improves navigation success rates, manipulation success rates, cross-environment generalization, or Sim-to-Real performance.

Assessment: This Is Not a Camera Business, but a Contest for the Gateway to Robotics Data

Kosmo One’s most marketable selling points are its 8K resolution and its ability to reconstruct 100 square meters in 15 minutes. What truly deserves attention, however, is Hesai’s first attempt to turn real-world spatial capture into a repeatable data supply chain.

If this pipeline works, robotics companies will no longer need to build all their own scanning teams, while shopping malls, factories, and hotels will be able to convert their spaces into continuously updated digital assets. Real 3D data will no longer be a “luxury” purchased for a small number of projects, but will increasingly resemble a standardized raw material for model training.

Yet before Kosmo can become industry infrastructure, it must progress from “capable of reconstruction” to “usable for training,” and then to “proven effective in training.” Visual quality can be showcased on a large screen at a product launch, but the geometric reliability, semantic completeness, and cross-tool compatibility of data assets can only be validated in real development workflows.

A more accurate assessment of Hesai’s launch, therefore, is that the direction matters more than the hardware itself. Kosmo addresses the data gap in embodied AI and offers a relatively comprehensive integrated hardware-software solution. But whether it can establish a defensible moat will not depend on how clearly the camera captures images. It will depend on whether Kosmo World can reliably transform hundreds or thousands of real-world spaces into training sets that developers are willing to use over the long term.

The next stage of competition in robotics may not be determined solely by who has the larger model or more joints. Whoever can copy the real world into training environments at lower cost will be more likely to control the data gateway for physical AI.

References

• ITHome: World’s First 8K Ultra-HD AI Spatial Camera, Hesai Kosmo One, Unveiled—Information from the launch event, including Kosmo One’s hardware specifications, reconstruction efficiency, Kosmo World platform capabilities, and pilot partnership with Amap.