Om AI Unveils VLX: Stuffing Multimodality into the Brain of Cameras

From June 27 to 29, Om AI released models on GitHub for three consecutive days, rolling out the entire VLX edge-side streaming multimodal series at once: VLX-Flow, VLX-Seek, and VLX-Go, three models with distinct roles covering parameter scales from 0.6B to 10B. The suite calls itself “the world’s first edge-side streaming multimodal model for the physical world.” The marketing language is ambitious, but when you break it down, it genuinely addresses a question that few people have tackled head-on over the past year: what should multimodal models actually look like when running on real devices?

Over the past two years, video understanding models have mostly been solving one problem: given a video, answer questions about it. Frames are sampled, encoded, inferred on, and output in one batch. This paradigm works fine for short-video summarization and content retrieval, but cameras do not wait for uploads to finish before seeing, and robots do not stand still waiting for questions. Input from the physical world is a continuous stream, not a file.

What Om AI has done this time is essentially transform video understanding from an “interface” into a “resident process.”

How the Three Models Divide Their Roles

First, let’s clarify the architecture. VLX is not one giant all-in-one model, but three specialized components corresponding to a complete “perception–localization–action” loop:

• VLX-Flow: Responsible for continuous perception and text interaction. Video arrives as continuous small segments, and the model maintains a reusable cache state through incremental encoding. When new frames arrive, it does not recompute historical context. By the time a user asks a question, the model has already been “watching for a while” and can answer directly.
• VLX-Seek: Responsible for precise localization. There’s a clever design here—it does not force the model to output coordinate numbers directly (LLMs generating coordinates has always been unreliable). Instead, coordinate generation is turned into region retrieval: selecting one region from a set of candidates. This compresses open-ended spatial reasoning into a multiple-choice problem over a closed set, dramatically improving reliability.
• VLX-Go: Responsible for action. Instead of outputting textual suggestions, it directly generates short-term waypoints and motion trajectories consumable by robots. Tasks like following, obstacle avoidance, and navigation are handled internally from “visual understanding” all the way to “motor commands.”

Together, the three models form a pipeline of “watching, locating, and moving” simultaneously. This is conceptually similar to past VLA (Vision-Language-Action) models, but Om AI splits the system more finely and aligns it better with the computational realities of edge devices.

What Actually Makes Streaming Multimodal Different From Traditional Approaches

This is the most worthwhile section to unpack.

Traditional video understanding generally follows two routes:

• Full-frame input: Hundreds of frames are fed into the model at once. Information density is high, but compute cost and latency become unsustainable for long videos.
• Fixed sampling: One frame is sampled every few seconds. Cheap, but easily misses motion details.

Neither works particularly well in camera scenarios, because critical events may happen between sampling intervals.

VLX-Flow’s approach is to split video into continuous short segments (for example, one every 2 seconds) and feed them sequentially into the model. The vision encoder converts new segments into representations usable by the language model, while the language model maintains a KV cache + recurrent state. New segments only trigger incremental updates rather than recomputation of the entire history.

The language model incorporates a Linear Attention module, which is the key step. Standard self-attention causes KV cache growth to expand linearly with sequence length, so running out of memory on long videos becomes inevitable. Linear Attention compresses history into a fixed-size recurrent state, allowing the video stream to continue indefinitely without memory usage continually increasing. The TTFT (time to first token) metric illustrates this particularly well:

• Full Attention: TTFT keeps increasing as the number of input images grows
• SlideWindow: TTFT fluctuates in a sawtooth pattern as windows fill and reset
• VLX-Flow: Through dual-layer memory compression of historical states, TTFT remains stable even for long sequences

The official single-stream latency figure is 0.06 seconds. In an edge-device context, that number is meaningful—it means the model can keep up with normal frame-rate video streams without downsampling or batching, processing camera input as it arrives.

Dual-Layer Memory: Preventing Text History and Visual State From Conflicting

Another interesting design in VLX-Flow is its dual-layer memory mechanism.

The first layer is the visual cache, which preserves details from recent time windows: actions, positions, object states, and short-term changes. The second layer is the text continuity layer, responsible for continuous descriptions, user questions, and model responses across long-term semantics. During multi-turn interactions, the text continuity layer synchronizes with the visual cache during merging, trimming, and replay operations, preventing mismatches such as “the text says there’s a cup on the table, but in the model’s internal visual memory the cup has already been removed.”

This problem becomes especially frustrating in real-world usage. A common failure mode in multi-turn dialogue is that the model remembers what it said previously but completely forgets that the visual input has already changed. Om AI uses a synchronization mechanism to bind these two memory chains together, which is a grounded engineering approach.

Why This Is Not “Shrinking a Cloud Model”

The company repeatedly emphasizes one statement: VLX is not a cloud model compressed down for edge deployment, but a system architected specifically for edge devices from the ground up. At first glance this sounds like marketing language, but looking at the technical details, the distinction is real.

You can quantize cloud multimodal models (such as GPT-4V or Qwen-VL), reduce parameters, or distill them below 1B parameters, and they will technically run—but their architecture is still fundamentally designed for “offline one-shot inference.” KV cache explosion remains a problem, frame sampling still drops information, only now with lower numerical precision and cheaper operators.

VLX changes the paradigm starting from the data construction stage. During training, it uses an “observe first, ask later” setup. Video segments stream into the model first, generating Stream Memory. Questions are then asked tens of seconds or even minutes later, and the model must answer based solely on its compressed memory state—it cannot rewind the video. This training objective means the model’s internal state is optimized for continuous operation rather than single-response inference.

This is also why the parameter scale starts at 0.6B. Edge scenarios impose hard constraints on compute and memory. Sub-1B models can run directly on drones, robot controllers, and AI PCs, while the 10B version leaves room for automotive and edge workstation deployments.

My Assessment: This Moves in the Right Direction

Over the past two years, multimodal models have achieved impressive benchmark scores, but deployment on real devices has remained awkward. Camera manufacturers want to integrate large models, only to discover that cloud latency cannot support real-time alerts. Robotics companies want to build VLA systems, only to find edge deployment infeasible, while cloud deployment cannot leave the factory floor. The industry has been missing a model architecture designed natively for continuous video streams.

VLX may not be the final answer, but it frames the problem clearly: unless the three constraints of continuous input, dynamic environments, and limited resources are solved, physical AI will remain stuck at the demo stage. Om AI’s decision to split perception, localization, and execution into three collaborating models is more pragmatic than forcing every capability into one giant model. Each model focuses on its own task, training objectives remain clean, and edge deployment becomes easier to modularize.

There are still several things worth watching:

• Real deployment performance: The 0.06-second latency is an official figure, but developers still need to benchmark the actual hardware, resolution, and historical context lengths involved.
• VLX-Seek’s region retrieval: The idea is strong, but candidate-region generation methods and precision limits will determine whether it can replace traditional detection heads.
• VLX-Go’s generalization ability: Direct waypoint output implies tight coupling with specific robot hardware. Cross-platform transferability may become an issue.
• Ecosystem compatibility: Integration with ROS, Isaac, and various embedded inference frameworks.

Om AI itself is not a lightweight player. CEO Zhao Tiancheng has received the Wu Wenjun Artificial Intelligence Science and Technology Progress Award, and the company already has visibility in Zhejiang’s embodied AI ecosystem. At AWE 2026, it showcased the concept of a “general intelligent terminal brain.” Releasing code continuously over three days with synchronized GitHub updates signals an intention to make VLX usable for developers, rather than merely presenting slides at a conference.

The Division of Labor Between Cloud and Edge

One final point about paradigms.

VLX takes the edge-device route, emphasizing local execution on hardware. Meanwhile, cloud multimodal models (GPT-5, Claude, Gemini, and others) continue advancing toward more complex reasoning, longer context windows, and broader general capabilities. These two directions are not in conflict. A robot can naturally use VLX-Flow for continuous local perception and VLX-Go for movement handling, while escalating complex decision-making problems to cloud-based large models when needed.

Cloud API aggregation services (such as platforms that let one API key access GPT, Claude, Gemini, DeepSeek, and others) solve the “complex reasoning in the cloud” part of the equation, while edge models like VLX solve the “real-time local perception” part. Future AI systems for the physical world will most likely combine both layers.

Multimodal models are evolving from “image question answering” toward “continuous perception.” VLX puts that shift front and center—cameras do not look at the world only once every five seconds, and robots cannot open their eyes only when questioned. Whether this architecture is worth betting on will depend on the empirical results over the coming months, but the direction is correct.

References

• Om AI GitHub Organization Page - Official release location for VLX series model code and documentation
• VLX Model Repositories - Model weights and configuration files on Hugging Face