Vivix Brings Video Generation into the Real-Time Era

Real-Time Multimodality Is No Longer Just Hitting Fast-Forward on Video Generation

Lingdong Shike recently officially unveiled Vivix, its first real-time interactive model. According to publicly available information, the model uses a unified streaming architecture that places multimodal understanding, state updates, and content generation within a single continuously running pipeline, achieving throughput of more than 10,000 video tokens/s on a single GPU.

This is a noteworthy figure, but one that also needs to be examined carefully.

Over the past two years, competition among video generation models has primarily focused on visual quality, duration, resolution, and prompt adherence. A user enters a sentence, the model computes in the background for tens of seconds or even several minutes, and ultimately delivers a complete video. In essence, it behaves more like a render farm with somewhat inconsistent performance: a request goes in, a finished video comes out, and the intermediate process is largely impossible to influence.

Vivix aims to change this interaction paradigm. Rather than merely generating a video file faster, it continuously updates its internal state and streams the next segment of content while receiving an ongoing flow of images, video, speech, or control signals. Users can speak, change actions, or move the camera midway through the process, and the model can immediately respond instead of discarding the previous result and regenerating everything from scratch.

In other words, traditional video models solve the problem of “make a video for me,” while real-time interactive models aim to solve “keep performing with me.”

What Exactly Is Unified in a “Unified Streaming Architecture”?

Real-time interaction does not simply mean reducing a process that originally took 30 seconds to three seconds. If the model must re-encode the context and recompute historical frames every time it receives a new instruction, the interaction will still be riddled with pauses, no matter how fast each individual inference may be.

The core value of the “unified streaming” architecture emphasized by Vivix should be reflected in three areas.

First, inputs are no longer divided into isolated requests

In a traditional invocation pattern, a video, a prompt, and a generation task usually form a complete request. Once the request ends, the model's computational state is released as well. The next round of interaction must either re-upload the historical content or rely on an additional memory module to reconstruct the context.

A streaming model, by contrast, must remain continuously online like a video conference. Each time the camera sends a new batch of frames or the microphone sends a new audio segment, the model processes only the newly added information while retaining the temporal state formed previously. The challenge is not “understanding an image,” but still knowing where a person is standing, what they just did, and why their clothes cannot suddenly change in the next frame after hundreds or even thousands of time steps.

Second, understanding and generation cannot operate independently

Many multimodal systems appear capable of real-time operation, but are actually built as chains of multiple models: a vision model recognizes the scene, a large language model makes decisions, and a video model generates the result. Speech recognition, speech synthesis, and motion control modules may also be inserted along the way.

Such pipelines are easy to build, but they accumulate latency. Each model must wait for the previous stage to produce output, and information may also be lost because the models use different representations. For example, after a vision model compresses a subtle movement into a textual description, it is difficult for the downstream video model to reconstruct the movement's speed, range, and spatial relationships.

The goal of a unified architecture is to let different modalities share lower-level temporal representations. The model does not need to first “translate” a video into a sentence and then generate video based on that sentence. Instead, it can directly establish relationships among continuous visual tokens, audio tokens, and control tokens.

This is also the most fundamental difference between a real-time interactive model and a “vision model plus video model” combination.

Third, output must be delivered as it is computed

Streaming output is already common in text models: the model generates one token, and the client displays one token. Video, however, is far more complex. A single image frame may correspond to a large number of discrete tokens, while character, lighting, camera, and physical motion consistency must also be maintained across frames.

If a model must wait until an entire segment has been computed before decoding it in order to preserve overall quality, it is still not a real-time system. True streaming video generation must begin outputting as soon as partial results are ready for playback, while ensuring that subsequent frames connect naturally.

This is like laying tracks while the train is already moving forward. Throughput is only the foundation; scheduling, caching, and state consistency determine whether the train derails along the way.

10,000 Video Tokens/s on a Single GPU Is Fast, but It Does Not Automatically Translate into a Real-Time Experience

The most eye-catching metric in the Vivix announcement is throughput exceeding 10,000 video tokens/s on a single GPU. It indicates that the team has made fairly aggressive optimizations somewhere in the video token encoding, inference, or decoding pipeline. Without complete test conditions, however, this figure cannot be converted directly into a frame rate or compared directly with other video models.

That is because different models may define “one video token” in entirely different ways.

Some models compress a frame into dozens of tokens, while others require hundreds or even thousands. Some tokens contain only low-resolution structural information, while others also carry texture and motion details. Just as “processing 10,000 characters per second” does not tell us whether the system is handling Chinese, English, or compressed binary data, video tokens/s only becomes fully meaningful when considered alongside the tokenizer's compression ratio, output resolution, frame rate, and precision settings.

At a minimum, developers still need to know the following:

• Which generation and model of GPU “single GPU” refers to, and how much VRAM it has;
• Whether 10,000 video tokens/s refers to encoding, backbone inference, decoding, or the entire end-to-end pipeline;
• Which numerical precision was used in testing, and whether quantization, sparsification, or speculative decoding was involved;
• The corresponding output resolution, frame rate, concurrency level, and context length;
• Time to first frame, instruction activation latency, and state drift after prolonged operation;
• Whether higher throughput results in a noticeable decline in visual quality or temporal consistency.

For real-time applications, average throughput may not even be the most important metric. A system that averages 60 frames per second but freezes for half a second every few seconds will feel worse than one that maintains a stable 30 frames per second. What truly determines the sense of interactivity is P95 or P99 latency, time to first frame, output jitter, and the ability to interrupt generation midway.

Therefore, 10,000 video tokens/s is a meaningful engineering signal, but not yet a complete product verdict. If Vivix's next step is to publish end-to-end data on standardized hardware and release reproducible demos, the achievement will be much easier for developers to include in their technology evaluations.

Video Generation Is Crossing the Boundary Beyond “Offline Tools”

The significance of Vivix is not that it is yet another video model, but that it is targeting a different market.

Today's mainstream video generation products are better suited to advertising assets, film and television previsualization, short-form video, and creative design. These scenarios can tolerate waiting and allow for manual selection and post-production corrections. Real-time interactive models, by contrast, are aimed at digital humans, game characters, livestreaming content, remote companionship, virtual teachers, and embodied intelligence simulations. They all share one requirement: the model must not only generate good content, but also respond at the right time.

Consider virtual livestreaming. An offline video model can generate a visually impressive host animation, but if a viewer suddenly asks a question, it cannot make the character turn its head, pause, and respond within a few hundred milliseconds. Traditional digital human solutions generally rely on fixed skeletons, prebuilt animations, and lip-sync control to ensure low latency. Their advantage is stability; their disadvantage is an obvious ceiling on expressiveness.

If models such as Vivix can bring generation quality and real-time control into the same system, digital humans will no longer have to choose between “natural but slow” and “fast but stiff.”

The same logic applies to games. Today's non-player characters primarily rely on scripts and state machines to act. Large language models can make them speak more naturally, but struggle to modify the entire visual world in real time. In theory, a unified streaming model could simultaneously process player speech, visual state, and action controls, then generate continuous feedback. This is not about attaching a chat box to an NPC, but about allowing the model to participate in the evolution of the world state.

However, there is still some distance to go before such systems can truly enter production. If a real-time generation system runs for hours, will character identity, scene structure, and physical rules gradually drift? How will states be isolated when multiple users interact simultaneously? Can generation errors be rolled back? How can content safety reviews intervene without introducing noticeable latency? All of these problems are more difficult than producing a ten-second demo.

Unified Models Are Appealing, but Modular Systems Still Offer Practical Advantages

From a technical storytelling perspective, a unified architecture almost always looks more elegant than an assembly of models. Data flows within the same representation space, theoretically reducing information loss and cascading latency while making end-to-end optimization easier.

For developers, however, unification does not inherently mean usability.

Although modular systems have higher latency, their components are easier to replace, debug, and audit. If speech recognition fails, that model can be replaced independently. If video generation is too slow, the resolution can be reduced. If there is a problem with business rules, a validator can be added between the language model and the execution layer. When a unified model behaves abnormally, teams often find it difficult to determine whether the problem occurred during perception, reasoning, or generation.

Cost is equally important. Not every input justifies activating the full video generation pipeline. When the user is inactive, a virtual character can remain idle using a lightweight animation system; the generation model only needs to be invoked when complex interaction occurs. Pursuing end-to-end unification for its own sake may keep GPUs fully loaded at all times, ultimately trading low latency for an unaffordable bill.

A more practical deployment approach may therefore be to use Vivix not as the solution for every stage, but as the real-time generation core, while retaining traditional control, content moderation, caching, and fallback modules around it. The model would handle open-ended scenarios that scripts cannot cover, while deterministic systems would provide guardrails.

What Developers Really Need to Wait for Is APIs and Benchmarks, Not the Next Demo

So far, the publicly highlighted features of Vivix have focused primarily on its unified streaming architecture and single-GPU throughput of more than 10,000 video tokens per second. Determining whether it can become a usable foundation model will require more specific product information, including whether the model will be available for trial, whether it can be privately deployed, how long its context can persist, which input and output modalities it supports, and how pricing will be structured.

The API for a real-time model also cannot simply replicate the traditional request-response format. Developers need persistent connections, bidirectional data streams, interruptible generation, state recovery, timestamp synchronization, and backpressure control. If the client's network briefly fluctuates, should the server continue generating, pause, or discard some frames? If the user interrupts while the model is speaking, can the previous output stop immediately? These API details will directly determine whether the model can be used in real-world applications.

There is also the question of evaluation. The visual-quality metrics commonly used for video generation cannot fully capture real-time interaction. More valuable future benchmarks should include at least:

1. Responsiveness: The latency between an input event and the corresponding output change;
2. Controllability: Whether an instruction can take effect accurately within a specified time window;
3. Long-term consistency: Whether characters and scenes remain stable after prolonged generation;
4. Interruption and recovery: Whether the system can transition smoothly after an instruction changes midway;
5. Unit cost: How much computing power is consumed per minute of real-time interaction;
6. Concurrency efficiency: How throughput and tail latency change across multiple simultaneous sessions.

Without this data, 10,000 video tokens/s is more like an engine's RPM on a test bench. With it, developers can determine how fast the engine will actually go once installed in a vehicle.

Verdict: The Direction Matters More Than the Number

The most commendable aspect of the Vivix release is not that it produced an eye-catching throughput metric, but that it is explicitly pushing multimodal generation toward continuous, bidirectional, and interruptible real-time interaction.

Video models have historically emphasized generation over interaction: the visuals have become increasingly refined, but users still have to wait outside the prompt box. If a unified streaming architecture can operate reliably, it will transform video generation from a content production tool into an application runtime. At that point, the model's output will no longer be a video file, but an ever-changing visual world.

It is still too early, however, to draw definitive conclusions. Single-GPU throughput above 10,000 tokens per second requires more complete test conditions, the unified architecture must demonstrate long-term consistency, and the real-time experience must withstand network fluctuations, multi-user concurrency, and safety reviews. For developers, the key question is not whether a character in a promotional video can turn around instantly, but whether the system still remembers who it is after running continuously for an hour.

If Vivix can provide public benchmarks, usable APIs, and deployment cost data, it will become a competitive new entrant in the real-time multimodal market. If it offers only peak throughput without end-to-end data, it will remain, for now, an impressive engineering demonstration.

In any case, the next stage of competition in video generation is no longer just about “whose visuals look more realistic,” but about “who can keep up with people in real time.”

References

Note: Publicly available materials currently do not include a complete technical report sufficient to reproduce Vivix's reported metrics. The following links provide additional background on multimodal models, real-time agents, and high-performance inference infrastructure. They do not imply that Vivix uses the specific implementations of these projects.

• NVIDIA TensorRT-LLM: A high-performance framework for large-model inference and serving, useful for understanding throughput optimization techniques such as quantization, caching, batching, and multi-GPU inference.
• LiveKit Agents: A framework for real-time voice and multimodal agents that demonstrates issues in real-time applications such as persistent connections, streaming input and output, and mid-generation interruption.
• LLaVA: A representative open-source visual instruction model that provides useful background for understanding visual encoding, language model integration, and multimodal representations.