SenseTime U1 Pro Invitation Test Next Month: Building the Foundation for the Era of Intelligent Agents

SenseTime has revealed its next move.

On June 25, SenseTime’s SenseNova-U1 Pro was officially unveiled, positioned as “the industry’s first multimodal intelligent agent foundation with a native unified core of understanding · generation · action.” According to plan, this model will begin its invitation-only testing in July.

This is not a routine model iteration. From naming to positioning, SenseTime has made it clear that it is aiming to stake its claim in the intelligent agent infrastructure layer.

1. What Does “Native Unified” Really Mean?

Let’s break down that mouthful—“understanding · generation · action native unified.”

Over the past two years, the development paths of multimodal models have generally divided into two camps:

Splicing camp: First train an understanding model (e.g., one that can interpret images and videos), then connect it to a generation model (e.g., one that can draw pictures, generate videos), and finally string them together with some intermediate layer. Early combinations like GPT-4V + DALL·E 3 and parts of Gemini 1.0's architecture showed traces of this.

Native camp: Train the model from the pre-training stage to learn both understanding and generation simultaneously, without relying on later splicing. GPT-4o is a hallmark product of this route—input and output both run through the same neural network, delivering noticeably better response speed and consistency.

This time, SenseTime has added “action.”

By “action,” they mean the model can directly call tools, operate interfaces, and execute tasks. Technically, this is known as function calling or tool use, but SenseTime is expressing a deeper concept: instead of outputting JSON for an external program to execute, the model itself has planning and execution capabilities.

For example:

• A traditional multimodal model is like a “consultant”: you ask how to book a ticket, and it tells you the steps
• A model with function calling is like an “assistant”: it can help check flights, fill forms, but you must confirm each step
• SenseTime’s envisioned “intelligent agent foundation” is more like a “representative”: you say “book me the cheapest ticket to Shanghai next week,” and it plans, executes, and adapts on its own when problems arise

The difference among the three lies not in individual abilities, but in the gradient of autonomy.

2. Why Build an Intelligent Agent Foundation Now

The timing is interesting.

2024 is called “the first year of intelligent agents” — at least in presentations. But actual deployment is lackluster. Most intelligent agent products are still in the “impressive demo, unusable daily” stage, for three key reasons:

1. Base models aren’t reliable enough

The essence of an intelligent agent is “model as brain, tools as hands and feet.” But if the brain occasionally misinterprets user intent, plans infeasible steps, or omits critical actions during execution, the entire system collapses.

Current solutions add “guardrails”: multiple confirmations, human intervention, limiting the scope of operations. But this turns the agent into a “dumb agent,” significantly harming user experience.

2. Fragmented multimodal abilities

Real-world tasks often involve multiple forms of information. For example, “help input this invoice into the reimbursement system” requires:

• Understanding the invoice image (visual understanding)
• Understanding the reimbursement system interface (UI understanding)
• Filling forms, clicking buttons (action execution)
• Handling exceptions appropriately (reasoning and decision-making)

If these abilities are provided by different models stitched together via APIs, latency increases, errors accumulate, and experience suffers.

3. Tool calling is an “add-on” rather than a “native ability”

Most models’ function calling is added through fine-tuning, essentially teaching the model to output specific formats of text. This causes two issues:

• The model doesn’t truly “understand” what the tool does, it just outputs specific formats in certain situations
• For complex tasks requiring multi-step tool calling, the model’s planning ability is insufficient

SenseTime’s decision to launch U1 Pro now bets on a judgment: The bottleneck for deploying intelligent agents lies not in the application layer, but in the base model layer.

Instead of patching the upper layer, it’s better to redesign a model from the ground up “natively built for intelligent agents.”

3. What Benchmarking GPT-Image 2 Means

Multiple media reports note that SenseTime has clearly positioned GPT-Image 2 as its benchmark.

This is both a smart and risky choice.

GPT-Image 2 is OpenAI’s native image generation capability, integrated into GPT-4o this year. Its biggest feature is not generation quality (though quality is good), but seamless integration with conversation:

• You can ask it to draw anytime in conversation without switching tools
• It understands what you want based on conversational context, without complicated prompts
• Generated images can be edited further, with the model remembering previous versions

In other words, GPT-Image 2 proves: multimodal shouldn’t mean “multiple modes,” but “one model, multiple expressions.”

Benchmarking this means SenseTime’s U1 Pro aims to deliver such “native unified” experience, and to advance further in the “action” dimension.

The risk: OpenAI’s native multimodality rests on massive data, huge compute, and long-term accumulation. Whether SenseTime can truly achieve “native unified” technically rather than “deep splicing” remains to be seen in real tests.

4. SenseTime’s Intelligent Agent Layout

This isn’t SenseTime’s first mention of intelligent agents.

Public information shows SenseTime’s intelligent agent layout in three layers:

Infrastructure layer: Large model + unified training/inference platform

SenseTime has its own compute infrastructure and full-stack model training and inference capabilities — prerequisites for making an agent foundation. If model and inference platform come from different providers, optimization space is limited.

Model layer: SenseNova series

SenseNova is SenseTime’s large model brand, covering language, vision, and multimodal series. U1 Pro is the latest flagship, positioned as the agent foundation.

The “U” may stand for Unified or Universal, emphasizing multimodal integration.

Application layer: Products like “Little Raccoon”

SenseTime’s desktop intelligent agent product “Little Raccoon” serves over 20 million users, with over 10,000 enterprise clients — a significant number.

More importantly, user feedback feeds back into model training. The hardest part of intelligent agent products isn’t technology, but obtaining real-world failure cases — when do users quit? Which tasks fail? Which operations cause errors?

A product with 20 million users means SenseTime has a continuous data flywheel.

5. Technical Details Guesswork

SenseTime hasn’t disclosed U1 Pro’s technical details, but based on “native unified” positioning and industry trends, we can make reasonable guesses:

Architecture

Unified Transformer backbone network: Understanding and generation share most parameters, with modality adaptation only in input/output layers — mainstream approach to “native unified.”

Multimodal tokenizer: Image, audio, video converted into token sequences and mixed with text tokens. This allows true simultaneous processing of multiple modalities rather than separate merging.

Built-in tool calling module: Not just prompt engineering to output JSON, but architecturally designed tool calling ability. Possibly inspired by Toolformer, letting the model learn when/how to call tools during pre-training.

Training

Large-scale multitask pre-training: Besides traditional language modeling and image-text alignment, add significant “task completion” data (e.g., given webpage screenshot and target, output click location).

Reinforcement learning fine-tuning: Core agent ability is “achieve goals,” hard to optimize with supervised learning. RLHF or similar is almost mandatory.

Simulated environment training: Model practices tasks in simulated OS, browser, apps — heavy engineering but boosts practical ability.

Inference

Streaming execution: Agents shouldn’t wait to plan all steps before acting; plan and execute while adapting in real time — high demands on inference framework.

Multi-level caching: Agent tasks span multi-turn dialogues and large contexts. Efficient context management is a big engineering challenge.

Tool calling optimization: Reduce latency in calling external tools — possibly internalizing common tools into the model or designing more efficient protocols.

6. Competitive Landscape Analysis

The agent foundation track has many players, but few can truly compete.

International players

OpenAI: GPT-4o is currently the strongest multimodal foundation; Operator is its official agent product. Focus seems more on toC productization than toB agent foundations.

Anthropic: Claude 3.5’s Computer Use is a key exploration of agent direction — proving feasibility of models directly operating computers. Conservatism slows its pace.

Google: Gemini series has strong multimodal ability, but agent-direction productization isn’t obvious. Google seems more intent on integrating AI into its own products than building a general agent foundation.

Domestic players

ByteDance: Doubao model has large C-end reach, but agent-direction layout unclear. Advantage: traffic and scenarios; disadvantage: weaker technical foundation.

Alibaba: Tongyi Qianwen series has advanced, with good open-source ecosystem. Focuses more on the model itself, less storytelling about an agent foundation.

Baidu: Wenxin Yiyan iterates quickly, aided by search scenarios. Weaker brand in the toB market limits promotion of agent foundation.

SenseTime: Model ability may not be top-tier, but strength lies in full-stack self-research (compute + model + application) and toB DNA. If U1 Pro delivers “native unified” experience, it could win in enterprise agent markets.

SenseTime differentiation

By choosing “agent foundation,” SenseTime avoids head-on competition with GPT-4 and Claude in language ability, focusing instead on the new “action” dimension.

This is a clever strategy:

• Language ability gaps are hard to close quickly given OpenAI’s data and compute advantages
• “Action” is a new battlefield with no absolute leader
• China’s market has natural demand for domestic agent foundations (data compliance, speed, localized service)

Of course, a smart strategy doesn’t guarantee success — product results matter.

7. Enterprise Market Opportunities

Potential scenarios for U1 Pro:

1. RPA upgrade

RPA is a mature market but has clear limitations:

• Requires predefined steps; breaks when interfaces change
• Cannot handle unstructured input, e.g., “process this email”
• High maintenance cost; small process changes require reconfiguration

New-gen RPA based on multimodal agents can fix this:

• Model understands interfaces, unaffected by UI changes
• Understands natural language commands, no need for exact step definitions
• Handles exceptions without stalling

This is a sizable replacement market.

2. Enterprise assistant

Imagine a truly useful enterprise AI assistant:

• “Make a weekly report from last month’s sales data” — it logs into CRM, exports data, generates charts, writes document
• “Schedule next Wednesday’s meeting with Mr. Zhang” — it checks calendars, finds free time, sends email, creates meeting
• “This client’s contract is expiring, remind me to follow up” — it sets reminders, prepares relevant materials

Current enterprise AI assistants mostly “answer questions,” not “complete tasks.” Agent foundations can change that.

3. Industrial inspection upgrade

SenseTime’s strong suit is visual AI, with many industrial inspection deployments. Agent foundation can upgrade this:

• Not just “detect defects,” but “determine defect type, severity, possible cause”
• Not just “output report,” but “auto-record in system, trigger workflow”
• Not just “spot checks,” but “comprehensive analysis, pattern discovery, risk alerts”

Upgrading from “visual AI” to “visual agent” is a fundamental leap.

8. Invitation Testing Strategy Guesswork

Launching “invitation test” in July is worth pondering.

Why invitation not public test?

Possible reasons:

Limited capacity: Agent foundation inference costs are higher than typical chat models (longer context, more tool calls, complex planning). Early capacity growth allows only limited users.

Need deep cooperation: Agent value manifests in specific scenarios; simple API calls can’t verify effectiveness. Invitation allows deep collaboration with key clients.

Risk control: Agents directly operate systems and tasks — failures have more serious consequences than chatbots. Invitation phase keeps risk within controlled bounds.

Create scarcity: Marketing angle — limited invitation creates buzz and attracts quality users.

Likely invitees

High-priority potential invitees:

• Existing large SenseTime clients: Already using other SenseTime products; low switching cost
• RPA vendors: Could use U1 Pro as underlying capability to upgrade their product
• Top vertical industry players: e.g., finance, healthcare, manufacturing — can provide high-quality scenario feedback
• Developer community influencers: Help spread, discover issues, contribute best practices

9. Developer Considerations

If you’re a developer interested in U1 Pro, watch for:

API design

Agent foundation APIs will be more complex than typical chat models. Pay attention to:

• Tool definition format: describing a tool’s functions, parameters, return values
• Execution observability: seeing model planning steps, results
• Error handling: what happens on failure, how to roll back/retry
• Permission control: limiting scope of model operations to prevent unauthorized action

Context management

Agent tasks often need long context:

• User’s original instructions
• Current environment state (e.g., webpage screenshot, system info)
• History of operations
• Tool return results

Efficient context management and cost control are big practical issues.

Evaluation standards

Agent model evaluation differs from traditional models:

• Traditional: “Is the answer correct?”
• Agent: “Was the task completed?”

The latter is harder to evaluate — requiring real execution environments. Whether SenseTime provides standard benchmarks and tools will be worth watching.

Cost structure

Agent costs differ from chat models:

• Single dialogue may involve multiple model calls (planning, execution, checking)
• Longer context windows required
• Possible extra cost for tool calls

SenseTime’s pricing, subscription plans, and enterprise discounts will affect adoption.

10. Cold Thoughts

A few less optimistic points:

Agents aren’t mature yet

Despite everyone pushing agents, the tech is still early-stage. Those who’ve used Claude’s Computer Use or OpenAI’s Operator know they aren’t “ready” yet.

SenseTime, as a follower, faces high difficulty in making an agent foundation surpassing the leaders.

“Native unified” may be marketing jargon

“Native unified” sounds premium, but how much is actual architectural innovation versus deep engineering integration remains to be seen in technical reports.

Historically, many “native XX” claims turned out to be marketing.

Agent moat isn’t in the model

Even if U1 Pro is strong, agent product moats are more about:

• Tool ecosystem: how many tools are supported and their quality
• Scenario accumulation: how many scenarios have been validated, how many best practices exist
• Data flywheel: whether user feedback efficiently improves the model

These require long-term investment — one model release won’t solve them.

---

In summary, SenseTime SenseNova-U1 Pro is worth watching. It represents domestic large model makers’ exploration of intelligent agents and SenseTime’s attempt to find differentiated positioning amid fierce competition.

July invitation test — let's see the results.