ZhiYuan Teams Up with Six Universities to Launch FlagSafe: Combining Red Teaming, Blue Team Defense, and White-Box Analysis on One Platform

Over the past two years, large model security has been like a pot where everyone cooks their own dish. Academia focuses on jailbreak attack benchmarks, vendors build content-filtering gateways, and security companies sell red team services—but their data doesn’t interconnect and methods aren’t unified. ZhiYuan’s current effort aims to tie these loose ends together.

Recently, the Beijing Academy of Artificial Intelligence (BAAI), together with Peking University, Beijing University of Posts and Telecommunications, Beihang University, Shanghai Jiao Tong University, the Institute of Information Engineering (CAS), and the Institute of Computing Technology (CAS), officially launched the FlagSafe Large Model Security Platform. The platform brings together multiple frontier research projects on model security, centered on red team exercises, blue team defense, and white-box analysis, with the goal of covering the full chain of risk discovery — defensive governance — mechanism interpretation.

This is among the few domestic attempts to make “mechanism interpretation” a core direction in a security platform. It also marks the biggest difference between FlagSafe and engineering-focused approaches like 360’s Large Model Security White Paper — FlagSafe leans toward research and aims to explain why models get compromised.

More Than Just Another Red Team Tool

To understand FlagSafe, we should first look at the current layering of model security tools.

There are roughly three approaches in the industry:

• Black-box red teaming: Represented by collections of jailbreak prompts and automated attack frameworks (like PAIR, GCG, AutoDAN). The advantage is that they can target any model; the downside is their lack of interpretability—when attacks succeed, the reason why remains unclear.
• Blue team defense: Input/output filtering, intent classification, and safety alignment fine-tuning. Technically mature but essentially “patch-based,” meaning that attackers can often bypass defenses with simple variations.
• White-box interpretability: Analyzing the model’s internals through activations, attention maps, and feature visualizations. This has been popular in academia (Anthropic’s circuit tracing, sparse autoencoders, etc.), but it’s far from engineering practice and comes with a steep entry barrier.

FlagSafe puts all three aspects onto a unified platform. This means researchers can use one infrastructure to run attacks, evaluate defenses, and then probe the model’s inner workings for explanations. Such integration is uncommon—previous systems were typically either attack libraries (like HarmBench, JailbreakBench) or evaluation leaderboards, focusing on one single dimension.

What Each Direction Aims to Solve

Red Team Exercises: Automation Is the Key

Traditional manual red teaming can’t keep up anymore. Cutting-edge models now have hundreds of billions of parameters and thousands of interaction modes—using human-written prompts to hunt for vulnerabilities is inefficient. The projects integrated into FlagSafe likely focus on automated attack generation and multimodal attacks—the former addresses coverage, the latter tackles new attack surfaces brought by image, text, and speech inputs.

Attack paradigms are also evolving from simply “tricking models into inappropriate behavior” to more covert levels:

• Indirect prompt injection, targeting Agent scenarios where documents or webpages are read externally
• Function call hijacking, targeting Agents that use function calls or Model Context Protocol (MCP)
• Hidden instructions in long contexts, exploiting attention dilution within million-token windows

These require entirely new red teaming frameworks beyond traditional NLP security testing.

Blue Team Defense: From Patchwork to Governance

For the blue team, the keyword is governance, not just filtering. Filtering is reactive; governance implies strategy, auditing, and metrics.

A practical defense platform must answer several questions:

• How can attack success rate (ASR) be calculated fairly, given different prompt templates and temperature parameters?
• How can we quantify the trade-off between safety and usefulness? Over-defense leads to overly cautious models, and businesses won’t accept that.
• Can defense strategies themselves be adversarially attacked? For example, jailbreakers might train bypass models specifically against your filter.

By leveraging multiple academic labs, FlagSafe can provide more thorough methodological answers rather than fragmented interpretations.

White-Box Analysis: The True Highlight

White-box analysis is what truly sets FlagSafe apart. This means directly inspecting the model’s internal parameters, activations, and circuits to identify the mechanisms underlying “safe” or “harmful” behaviors.

If this works out, it would enable:

1. Defenses that no longer depend on external classifiers but instead remove harmful capabilities from within the model (unlearning)
2. Verifiable alignment effects, not just benchmark scores
3. Regulatory audits that could evolve from black-box testing toward white-box transparency

This approach is technically demanding—it requires access to model weights, training data, and inference infrastructure. As a nonprofit research institute collaborating with top national universities, ZhiYuan is more likely than commercial companies to push this forward.

Why Now

The release of FlagSafe in May 2026 comes with clear context.

On one side, model capabilities are accelerating exponentially—GPT‑5.5, Claude 4, and domestic giants like DeepSeek, Qwen, and Zhipu are diving into intelligent agent abilities, massively expanding the attack surface. OpenAI even offered large bounties for GPT‑5.5 biosecurity vulnerabilities, implying that top labs themselves are uncertain about their safety.

On the other side are growing compliance pressures. The International AI Safety Report released in early 2026 formally outlined a four-stage framework of risk identification—assessment—mitigation—governance. Domestic requirements for generative AI registration and safety reviews are also tightening. Enterprises wanting to deploy large models for serious business applications must have credible security evaluation systems.

A third factor is structural: domestic large model security research is scattered across university labs—there are many papers but few engineered or platformized outputs. FlagSafe acts as a “middleware hub” that aligns these decentralized research results with unified interfaces and evaluation datasets.

What It Means for Developers

For frontline developers, FlagSafe won’t immediately replace existing security solutions, but several aspects are worth attention:

• Open evaluation datasets: If FlagSafe later releases its red team prompt collections and evaluation scripts, they could be integrated into CI workflows for security regression testing of fine-tuned models.
• Tooling for white-box methods: Ready-to-use implementations for techniques like sparse autoencoders or activation guidance would be valuable for RAG or Agent teams diagnosing “why the model made this mistake.”
• Unified metrics: When explaining to clients “how secure our model is,” you could cite a recognized third-party platform score rather than invent your own.

Expected typical usage (based on disclosed directions):
  Red team side: call attack generation API → batch test target model → output ASR report
  Blue team side: connect defense middleware → collect interception stats → analyze false positive rates  
  White-box side: upload model weights → extract features → locate circuits responsible for harmful behavior

Points to Watch

While strengths are clear, some open questions remain. The information released about FlagSafe is still conservative, leaving several issues for later updates:

• How to handle closed-source models in white-box analysis? Commercial models won’t hand over weights—will the platform only serve open-source models, or use a federated scheme?
• Evaluation dataset contamination: Once evaluation sets are public, future model training data will likely include those prompts—can scores still reflect real security levels?
• Ecosystem collaboration: The six founding institutions are research-oriented, but real attack-defense expertise lies with industrial players like Alibaba Security, Tencent Zhuque, and 360. Whether FlagSafe can bring the industry into cooperation is crucial.

Large model security has long been a topic of discussion but little coordinated action. Those who act do so in isolation. FlagSafe might not solve everything at once, but at least it brings research, engineering, and evaluation to the same table. That alone is a necessary integration for China’s AI security ecosystem.

As for whether the platform will open-source its code, release APIs, or support mainstream models—let’s watch its next moves in the coming months. It’s worth paying close attention.

References

• Hugging Face — The main distribution hub for large model security datasets and open model weights; FlagSafe’s future releases are also expected to appear here.