Numerical Technologies Throws Out Vidu S1: The Video Generation Model Can Talk Now

On July 3, Numerical Technologies unveiled the Vidu S1 real-time interactive model. This time, the focus isn’t on image quality or duration, but rather on a direction few have seriously pursued before—real-time interactivity. You can talk to an AI-generated character as if you were making a video call—it listens to you and generates the upcoming visuals in real time.

Simply put, video generation is moving from “pre-made” to “live broadcast.”

This Is Not Just Another Text-to-Video Model

Over the past year, the video generation track has been dominated by the “batch” players like Sora, Runway, Keling, and the Vidu Q series—you provide a prompt, wait tens of seconds to a few minutes, and get a video that lasts a few seconds to over ten seconds. Everyone is competing in quality, length, and controllability, but interactivity has always been nonexistent: once a video is generated, it’s a finished product. You can’t interrupt midway, change the narrative, or make characters respond to you.

Vidu S1 aims to tear down that wall. The official positioning calls it “for real-time interactive scenarios,” with several key metrics:

• 540P (960x540) resolution, 25FPS frame rate, up to 42FPS at maximum
• Unlimited-duration continuous interaction, not a pre-generated video but continuous generation
• Supports voice-controlled narratives, the on-screen character acts based on what you say
• Customizable initial appearance and voice, from real people to anime and cute pets

The most critical number here is 25FPS. To achieve a “video call” experience, the frame rate has to hold up real-time responsiveness; otherwise, users would be watching a slideshow, not a conversation.

Technical Approach: The AR + Diffusion Combo

Vidu S1 adopts an autoregressive diffusion (AR + Diffusion) approach, which is worth breaking down.

Traditional video diffusion models (like the Sora type) “denoise an entire video sequence from noise all at once”—you give a prompt, and the model calculates all frames in the latent space at once, outputting a finished product. This approach yields great visual quality and global consistency but is inherently at odds with “real-time.” You can’t inject new instructions halfway or generate and play simultaneously.

Vidu S1 does the opposite: it continuously predicts and generates subsequent content based on previously generated frames, current voice commands, and dialog context. It’s more akin to a large language model’s next-token prediction process, except it predicts the next segment of the video. Past frames serve as context, voice input becomes an on-the-fly prompt, and the model advances step by step along the time axis.

The advantages of this approach are clear:

1. Supports streaming playback—it can play as it generates, no need to wait for a full clip to finish
2. Responds instantly to new commands—say “turn around,” and the next segment will do so
3. Theoretically unlimited duration—not constrained by fixed windows

The trade-offs are equally real: maintaining global consistency is harder, errors accumulate over time (the longer it goes, the more “drift” you get), and computational cost is high—it has to produce 25 frames per second in real time. Meta’s MovieGen and some of Runway’s new research are exploring similar directions, but when it comes to delivering a product-grade real-time interactive model, Vidu S1 seems to be ahead.

From Q1 to S1: A Split in the Product Line

Those familiar with Vidu might recall that in March, Numerical Technologies launched Vidu Q1, focusing on “high controllability”—multi-subject control, sound synchronization, image enhancement, aimed at professionals making polished content. The later Q3 even topped Artificial Analysis’s rankings.

S1, however, is clearly on a different track: the Q series targets “productivity tools,” while the S series focuses on “real-time experiences.” This split is smart. Video generation has become heavily homogenized—everyone’s competing over resolution, duration, and cinematic language. But what can users actually do with the generated videos? Post shorts, make ads, edit footage—batch generation already meets those needs.

Where’s the real growth potential? In digital live streaming, virtual companionship, AI customer service, and educational interactivity—scenarios that require “conversation + imagery” simultaneously. That’s exactly what S1 targets. Imagine an AI character that can video chat with you in real time, move and change expressions based on your commands—it could mean a lot for companion apps, virtual streamers, and educational platforms.

Part of a Broader World Model Ambition

This move makes even more sense within Numerical Technologies’ overall strategy. In April, they completed a nearly 2 billion RMB Series B funding round led by Alibaba Cloud. The company’s narrative has already evolved from a “video foundation model” to a “general world model”—a digital-world WGM (World Generation Model) + a physical-world WAM (World Action Model, the same Motus framework open-sourced last December).

The S1’s ability to “understand real-time instructions and generate subsequent visuals” is essentially a core capability of a world model: predicting the next state based on historical states and external input. Replace speech input with a robot’s sensor signals, and the generated video with an action sequence, and the same architecture can be applied to embodied intelligence. So S1 isn’t just an interaction demo—it’s Numerical’s proof of the generalizability of the autoregressive + diffusion paradigm.

Real-World Experience and Open Questions

S1 is now in closed beta with two access points:

• Online experience: vidu.cn/vidu-stream
• API platform: platform.vidu.cn/live/landing

Based on the official demos, responsiveness and visual continuity look decent, but a few issues remain to be field-tested:

1. Actual latency: 25FPS is the generation rate, but the end-to-end delay—from when you speak to when the video responds—is what matters. Below 500 ms feels smooth; over 1 s starts to feel awkward.
2. Character consistency in long conversations: after 10 minutes, is the character’s face, hairstyle, and outfit still the same person? Autoregressive models notoriously struggle here.
3. Is 540P enough? As a real-time interaction compromise, it’s reasonable, but will look fuzzy on large screens. Commercial scenarios (like virtual streamers) will likely demand higher resolution.
4. Cost: 25FPS real-time diffusion inference isn’t cheap—scaling it up efficiently could be a challenge.

In a Nutshell

Vidu S1 may not be the prettiest video model, but it’s one of the first serious attempts to fuse “video” and “conversation.” Over the past year, the video-generation field has been circling around quality and length; S1 shifts the track—from “generating a video clip” to “generating a conversational live entity.”

If this path works out, sectors like virtual companionship, digital live streaming, and AI education—long hindered by “talking without visuals” or “visuals without interactivity”—could be reshuffled. Whether it will succeed depends on user feedback and iteration speed in the coming months.

References

• ITHome: Supports real-time video calls and voice control of video direction, Numerical Technologies releases Vidu S1 real-time interactive model — First coverage of the Vidu S1 launch, including technical details and specifications