Ant Lingbo Open-Sources LingBot-Video: Building a Physics-Aware Video Foundation Model for Robots

On July 9, Ant Lingbo Technology open-sourced two models at once: LingBot-World 2.0, aimed at real-time interactive world generation, and LingBot-Video, the world’s first open-source video generation foundation model designed for embodied intelligence. The former integrates Agent mechanisms into a world model, enabling hour-long real-time generation at 720p/60fps. But the truly eye-catching release is the latter—it attempts to answer a long-unresolved question: what should a video foundation model built for robots actually look like?

Over the past two years, video generation has largely been dominated by the “digital content creation” path represented by Sora, Veo, Kling, Wan, and Seedance, where models compete on visual quality, camera movement, and character consistency. But when these models are directly used as “world simulators” for robots, the problems become obvious immediately: physically implausible behavior, blurry motion details, poor task completion, and slow inference. LingBot-Video takes a straightforward approach: don’t make movies—make videos robots can understand.

From Dense to MoE: Making Video Foundation Model Parameters “Actually Work”

The core architectural decision behind LingBot-Video is replacing the mainstream DiT+Dense setup with DiT+MoE. The model has 30B total parameters, but only around 3B are activated during a single generation run. According to official figures, inference efficiency is roughly 3× higher than Dense models of similar scale.

This choice is particularly interesting. The video generation field has traditionally been cautious about MoE, mainly due to concerns around temporal consistency and expert routing stability. Image/video token counts are one to two orders of magnitude higher than in language models, amplifying MoE load balancing issues. Companies behind Wan, Seedance, and Cosmos still primarily rely on Dense architectures. One practical reason Lingbo was willing to move first in embodied scenarios is that robotic applications are extremely sensitive to inference latency. In VLA training, video foundation models are frequently invoked during rollout stages, and the inference cost of a Dense 30B model is nearly unacceptable. Sparse activation in MoE effectively decouples “model capacity” from “inference cost,” making the trade-off well-suited for embodied applications.

Based on public evaluations, LingBot-Video achieved an overall score of 0.620 on RBench, an embodied video evaluation benchmark jointly released by Peking University and ByteDance, outperforming Wan2.6, Seedance 1.5 Pro, and Cosmos3 Super. It’s important to note that RBench specifically measures performance on embodied-relevant dimensions such as physical plausibility, action executability, and task semantics, rather than the aesthetic quality emphasized in traditional video benchmarks. Lingbo’s internal benchmarks compared the model against five open-source models including Cosmos3 and Wan 2.2, claiming “systematic leadership” in embodied intelligence scenarios.

70,000 Hours of Embodied Data: The Real Moat

If MoE is the architectural differentiator, then the model’s data profiling engine is the truly difficult part to replicate.

On top of internet video data, LingBot-Video additionally incorporates three categories of embodiment-related data:

• VLA (Vision-Language-Action) data: multimodal records of robots performing real-world tasks, including action labels
• VLN (Vision-Language-Navigation) data: trajectories of robots moving and navigating through real environments
• Ego first-person data: including human first-person operation videos used to supplement dexterous manipulation scenarios

The total embodied dataset reaches 70,000 hours. For comparison, the Open X-Embodiment dataset used in early RT-2 work amounted to only several thousand hours. While internet-scale general video datasets are large, they provide almost no annotation value for problems like “how a hand grips a mug handle” or “how a robot foot slips on grass.”

More importantly, the data profiling engine performs hierarchical balancing. Rather than simply mixing embodied data into general video corpora, the pretraining curriculum was redesigned around core robotic needs such as physical interaction, action causality, and first-person spatial relationships. This explains why the model can pull ahead on RBench: general-purpose video models learn “what videos should look like,” while LingBot-Video learns “what rules the events in videos should obey.”

What Problems Do Embodied Video Foundation Models Actually Solve?

For developers unfamiliar with this area, it’s worth expanding on the background. Video generation models mainly serve three purposes in embodied intelligence:

1. World Models: Given a current state and action sequence, predict future frames. Used for planning, model-based RL, or safety simulation.
2. Data Generators: Real-world robot data is extremely scarce and expensive. Many teams currently use video foundation models to synthesize rollout data for VLA training.
3. Reward / Verifier Models: Determine whether an execution video successfully completed a target task. The semantic understanding capabilities of video models can serve as general-purpose reward signals.

All three tasks prioritize “correct physics and motion” over visual beauty. This is also why NVIDIA’s Cosmos series adopted a different technical route from the beginning—injecting physical simulation priors into training. LingBot-Video takes a more “data-driven” approach: instead of relying on explicit physics engines, it aims to let the model learn physical consistency from massive embodied datasets.

Judging from the RBench scores, this approach appears viable for now. Of course, a score of 0.620 is still far from “robots can directly rely on it for planning.” But as an open-source foundation model, it significantly reduces the workload for downstream research teams—in the past, teams had to collect tens of thousands of hours of robot data themselves; now they can directly fine-tune on top of this base model.

How Does It Relate to LingBot-World 2.0?

Since both models were open-sourced on the same day, it’s worth clarifying their roles.

• LingBot-World 2.0 focuses on real-time interactive world generation: users or Agents provide actions or instructions, and the model evolves the environment in real time. It resembles a “playable simulator,” emphasizing 60fps real-time responsiveness and long-horizon stability (hour-level). By introducing Agent mechanisms into world models for the first time, the world is no longer passively reactive but can continuously evolve through internally driven Agents.
• LingBot-Video focuses on offline video foundation model capabilities: given a video prefix or action sequence, it generates high-quality, physically plausible continuous video as a representation foundation for downstream embodied tasks.

Simply put, the former is a “real-time engine,” while the latter is a “pretrained base model.” The two can be combined—LingBot-Video provides representations and physical common sense, while LingBot-World 2.0 provides interaction and real-time capability. This roughly corresponds to the convergence within Lingbo of Yann LeCun’s JEPA-style world models and interactive world model approaches like Genie.

Some Questions Still Without Answers

The model has been open-sourced, but several aspects remain worth watching:

• How well does MoE maintain long-term consistency in video generation? Does the 3× inference speedup come at the cost of reduced stability? In generations longer than 30 seconds, will expert drift cause abrupt visual changes? The community will need more long-video testing to answer this.
• The composition and distribution of the 70,000-hour embodied dataset. If more details about the proportions, sources, and cleaning strategies of the VLA/VLN/Ego datasets are disclosed, it would be a major contribution to the embodied AI community. The current technical report reveals too little.
• Comparison with the Cosmos series. NVIDIA Cosmos follows a hybrid “physics engine + generative model” route, while LingBot-Video is purely data-driven. Which approach is more scalable in the medium to long term remains a genuinely open question.
• Empirical results in downstream VLA training. No matter how strong the base model is, what ultimately matters is whether robot policies trained with it achieve higher success rates. This likely won’t be validated until the first wave of community users reports results.

Final Thoughts

Over the past year, the strongest momentum in China’s video generation space has come from Kling, Wan, and Seedance, all focused primarily on content creation. Ant Lingbo’s move directly into embodied AI avoids the crowded red ocean and occupies a niche where no open-source foundation model previously existed. The label “world’s first” is sometimes just marketing language, but in this case, combined with the MoE architecture and 70,000 hours of embodied data, the claim appears credible.

For developers working on robotics and VLA, it’s worth paying attention to the real-world performance after the model weights are released—especially differences in training efficiency and policy success rates when replacing Cosmos or Wan in world-model rollouts. That is where the true value of an embodied video foundation model will ultimately be tested.

References

• 人人都是产品经理 Twitter Update - Summary of LingBot-Video release information (VPN required)