DeepSeek Releases Visual Primitive Thinking Framework: Enabling Multimodal Models to "Point" While Reasoning

DeepSeek, in collaboration with Peking University and Tsinghua University, has published a paper titled Thinking with Visual Primitives and simultaneously open-sourced the complete code repository. This framework accomplishes something that sounds simple but has never been done well before—it allows multimodal models, during their reasoning process, to directly “point” to specific locations in an image, much like how humans point at things while speaking.

Not “look, then answer,” but “think while pointing.”

Core Idea: Coordinates as Part of the Thought Process

Traditional multimodal reasoning models usually process images by “looking once, remembering, and then reasoning with text.” Once image information enters the language reasoning chain, it becomes purely textual descriptions, losing much of the spatial information. If you ask such a model to count the red squares in a picture, it might miscount; if you ask whether object A is to the left or right of object B, it might guess.

The approach of Thinking with Visual Primitives (TwVP) is to treat spatial markers—specifically, coordinate points and bounding boxes—as the minimal thinking units within the reasoning chain.

What does that mean? Traditional Chain-of-Thought reasoning looks like this:

Step 1: There are three objects in the image, namely...
Step 2: The red square is to the right of the blue circle...
Step 3: Therefore, the answer is...

Whereas the chain in Visual Primitive Thinking looks like this:

Step 1: I see an object [point: (234, 156)], which is a red square
Step 2: Another object [bbox: (45, 89, 120, 200)] is a blue circle
Step 3: [point: (234, 156)] is to the right of [bbox: (45, 89, 120, 200)]
Step 4: Therefore, the answer is...

Coordinates are no longer just the output of reasoning but anchors within the reasoning process. At each step, the model can “point back” to the image to confirm which object it’s referring to and where it is. This design is intuitively reasonable—humans also analyze complex images by looking, pointing, and thinking, rather than shutting their eyes and recalling from memory.

Technical Implementation: How to Teach the Model to “Point”

The framework’s technical implementation has several key layers.

Definition of Visual Primitives

The paper defines two types of visual primitives:

• Point: A single location in the image, represented as (x, y), suitable for marking object centers or keypoints.
• Bounding Box (BBox): A rectangular region represented as (x1, y1, x2, y2), suitable for delineating object extent.

These two primitives cover most spatial reasoning scenarios. Points answer “where,” bounding boxes answer “how large.”

Training Strategy

Models don’t naturally insert coordinates into their reasoning chains. The paper presents a specialized training pipeline that constructs reasoning data annotated with visual primitives. Each reasoning step in the training data may contain explicit references to image positions. Through learning from these examples, the model gradually acquires the ability to “refer to spatial positions during reasoning.”

Notably, the current version requires explicit trigger words to activate this capability. In other words, you have to specify in the prompt, “please use visual primitives for reasoning,” for the model to use it. It cannot yet autonomously decide when to “think while pointing.” This limitation is explicitly stated in the paper.

Compatibility with Existing Architectures

The framework doesn’t make drastic architectural changes. Instead, it extends the token vocabulary and adjusts training data on top of existing multimodal models. Coordinate information is encoded as special token sequences and processed in the same sequence as text tokens. This design makes it relatively easy to adapt to other multimodal models.

Evaluation Results: A Clear Lead in Topological Reasoning

The paper evaluated the framework on seven public benchmarks and four custom benchmarks, comparing it with models such as GPT-5.4, Claude Sonnet 4.6, and Gemini 3 Flash.

Overall result: An average score of 77.2%, the highest among all tested models.

But the most interesting part isn’t the average—it’s the variance across tasks.

Regular Tasks: Steady Lead

In counting and spatial reasoning tasks, the Visual Primitive Thinking model achieved consistent leadership. This isn’t surprising—when the model can precisely tag every object’s location during reasoning, counting and spatial judgments naturally become more accurate.

Imagine counting red dots on a dense map—if you can mark the ones you’ve already counted, your accuracy improves. That’s what Visual Primitive Thinking does.

Topological Reasoning: A Striking Lead

The most remarkable improvement appears in topological reasoning tasks. The paper introduced two benchmarks: Maze Navigation and Path Tracing.

| Task | Thinking with Visual Primitives | GPT-5.4 | Claude Sonnet 4.6 | Gemini 3 Flash | |------|-----------------|---------|-------------------|----------------| | Maze Navigation | 66.9% | <51% | <51% | <51% | | Path Tracing | 56.7% | <51% | <51% | <51% |

All other state-of-the-art models performed below 51%—essentially random guessing. In contrast, Visual Primitive Thinking reached 66.9% and 56.7%.

What does this difference indicate?

Topological reasoning requires the model to understand connections and paths between elements in an image. These tasks are highly dependent on spatial information, and a purely textual reasoning chain is almost useless. Try walking through a maze with your eyes closed—just following verbal instructions like “turn left, then right” doesn’t get you far.

Visual Primitive Thinking allows the model to “point” back to specific maze locations during reasoning, tracking where it is and where it should go next—unlike other models.

It also reveals a reality about the field: mainstream multimodal models are nearly blank when it comes to topological reasoning. Not just weak, but almost incapable.

Why This Direction Matters

Visual Primitive Thinking addresses a real, practical shortcoming in multimodal AI applications—not just an academic curiosity.

Industrial Inspection and Medical Imaging

In industrial inspection, models must locate defects on complex circuit board images and explain why they’re defects. Traditional multimodal models might say “there’s a bad solder joint,” but not specify where. Visual Primitive Thinking can pinpoint defect coordinates in its reasoning, producing answers like: “At position (345, 678), the solder area is insufficient—40% smaller than the neighboring joint at (350, 690).”

Medical imaging is a similar case. Radiologists think while looking and pointing. An AI assistant that can “point” while reasoning is far more practical than one that outputs only textual conclusions.

Autonomous Driving and Robotics

Autonomous driving perception modules require spatial reasoning with precision. “There’s a pedestrian ahead” isn’t enough—the system must know the pedestrian’s exact position, motion direction, and relative distance to the vehicle. The Visual Primitive Thinking framework is inherently suited for such tasks involving continuous spatial tracking.

Robotics operation follows the same logic: executing “place the red cup on the left side of the table” requires maintaining consistent spatial reference to the cup and table throughout reasoning.

Document Understanding and Data Extraction

Interpreting complex documents such as blueprints, flowcharts, or infographics is also a spatial reasoning problem. Relationships such as connection, hierarchy, and flow are all encoded in spatial layout. Models that can "point" have a natural advantage in these tasks.

Limitations: Honesty Over Hype

The paper’s discussion of limitations is unusually candid—commendable in today’s “leaderboard-first” environment.

Resolution Bottleneck. Limited input image resolution sometimes leads to coordinate inaccuracies in fine-grained scenes. In simple terms, if objects are too small or dense, the model’s “point” might not be precise. This issue plagues all vision models but is particularly impactful for a framework that depends on spatial precision.

Explicit Trigger Requirement. As mentioned, the visual primitive reasoning ability currently must be activated via specific prompt keywords. The model cannot yet decide autonomously when to “think and point.” In real applications, this means either the user or an upstream system must judge when to use it, raising usability costs.

Limited Generalization. The coordinate-based topology reasoning skills still struggle with cross-domain generalization. Good performance in mazes and path tracing doesn’t guarantee equal success in all topological tasks.

All three limitations point in one direction: this is still a v1. The concept is solid, but it’s not plug-and-play yet.

Comparison with Competitors: A Different Approach

Different teams in multimodal reasoning take different paths.

OpenAI’s GPT series and Google’s Gemini series mainly improve multimodal capability by scaling up model size and training data, relying on implicit spatial learning. Anthropic’s Claude series excels in textual reasoning but hasn’t focused heavily on multimodality.

DeepSeek took a distinct path: rather than making models “guess smarter,” it gave them a new thinking tool. Coordinates and bounding boxes act as “fingers” for the model’s reasoning chain, allowing it to point to parts of the image mid-thought.

The strength of this approach lies in its composability. Visual Primitive Thinking doesn’t replace existing models but can be added as a capability module. Experiments show that for non-spatial tasks, introducing visual primitives does not harm performance.

The downside is also clear: this framework currently only tackles spatial reasoning. Multimodal understanding also involves temporal reasoning, causality, and commonsense reasoning—areas where Visual Primitive Thinking offers no special boost.

Open-Source Ecosystem: Code Ready-to-Use

The corresponding code repository has been open-sourced on GitHub: deepseek-ai/Thinking-with-Visual-Primitives.

The repository includes model weights, training code, evaluation scripts, and benchmark datasets—providing a complete foundation for teams seeking to replicate or extend the work.

DeepSeek has consistently pursued aggressive open-sourcing—from DeepSeek-V2 to R1 to the new Visual Primitive Thinking—and has released papers and code simultaneously each time. This strategy has strongly benefited the academic and developer communities, contributing to DeepSeek’s rapid community traction.

For developers integrating spatial reasoning into multimodal applications—especially in industrial vision, document understanding, or map navigation—this repository is worth close attention. Through platforms like OpenAI Hub and similar API aggregators, developers can conveniently compare DeepSeek’s performance against other leading models on their specific tasks and choose the most suitable solution.

One-Sentence Summary

The core contribution of Visual Primitive Thinking is proving that “maintaining explicit spatial references during reasoning” in multimodal models actually works—and works well for certain tasks. It’s not a universal solution that surpasses all models, but it highlights a neglected truth: multimodal reasoning should not discard spatial information.

The next steps will determine whether this framework can evolve from “manually triggered” to “self-activated,” generalize beyond mazes to broader topological contexts, and overcome its resolution limitations as vision encoders improve.

The answers to these questions will decide whether Visual Primitive Thinking remains just a well-written paper—or truly changes how multimodal AI thinks.

---

References

• Discussion thread on DeepSeek’s “Thinking with Visual Primitives” Framework — Technical analysis of the paper from the Linux.do community
• deepseek-ai/Thinking-with-Visual-Primitives — Open-source code repository accompanying the paper