133 Million Conversations Slipped Through the Safety Net

One of Anthropic’s key safety defenses was not actually working for nearly a year.

On August 14 local time, Anthropic disclosed in its latest safety report that traffic generated while external contractors provided human feedback for its models did not pass through the classifier designed to intercept biological-risk requests between May 2025 and April 2026. Approximately 50,000 people were affected, involving around 133 million conversations in total.

Anthropic said its internal investigation has so far found no evidence that these requests were actually used for abuse, and that it has tightened vetting requirements for external contractors and vendors.

But that does not mean the incident can be dismissed as an ordinary configuration error. The problem was not that a single dangerous prompt slipped through, but that an entire category of high-risk traffic remained in a blind spot of the safety controls for an extended period—and was not identified until nearly a year later.

133 Million Conversations Does Not Mean 133 Million Bioweapon Requests

First, it is important to clarify a fact that headlines can easily obscure.

Anthropic disclosed that approximately 133 million contractor conversations did not pass through the relevant biosafety classifier. It did not say that all 133 million conversations involved biological weapons, nor does it mean that the model produced dangerous content 133 million times.

These conversations came from a large pool of external workers and were primarily used for human feedback, model evaluation, and data collection. They may have covered a wide range of tasks, including ordinary Q&A, capability testing, safety testing, and alignment labeling. In the currently available public information, Anthropic has not disclosed what proportion of the requests actually involved chemical, biological, radiological, or nuclear risks.

However, a large denominator does not make the problem smaller.

External evaluators may be more likely to encounter aggressive test cases, including jailbreak prompts, tests of dangerous-knowledge boundaries, and explorations of model failure modes. If a category of traffic is more likely than ordinary user traffic to deliberately probe safety boundaries, it should receive stricter monitoring—not a more permissive exception path.

This is the most important point for developers to take away from the incident: unscreened traffic does not mean harm has already occurred, but it does mean the platform lost the ability to identify harm promptly.

A safety system cannot use “no incident was found afterward” to prove that “there was no prior risk.” The absence of an alarm may mean there was no fire—or it may mean the smoke detector was never powered on.

What Failed Was Not Model Alignment, but the Deployment Control Plane

Anthropic’s defenses against high-risk content do not rely solely on Claude learning to refuse requests during training. According to its publicly described mechanisms, the complete defense typically includes multiple layers:

1. Model training and alignment: Teaching the model to proactively refuse clearly dangerous requests;
2. Input classifiers: Analyzing user prompts to identify intentions such as weapons development or pathogen enhancement;
3. Output classifiers: Checking whether model responses provide actionable, executable dangerous information;
4. Real-time blocking: Rejecting requests, terminating outputs, or routing them into stricter handling processes once risk reaches a threshold;
5. Offline monitoring: Sampling, clustering, and analyzing logs for anomalous behavior;
6. Red teaming and bug bounties: Proactively searching for jailbreak methods, obfuscation techniques, and classifier blind spots.

The central issue in this incident was that contractor traffic did not enter the required biosafety classifier pipeline. In other words, the model itself may still have retained refusal capabilities developed through safety training, but the real-time checks deployed around the model did not cover these requests.

It is similar to a cloud provider configuring a WAF, intrusion detection, and audit logs for its production environment, only for a partner to use a separate gateway that bypasses most of those controls. The servers may not be compromised immediately, but the security team can no longer see what is happening through that entry point.

From an engineering perspective, this kind of problem is often more difficult than a decline in the accuracy of an individual classifier.

Classifier misjudgments at least leave behind invocation records, risk scores, and enforcement outcomes, allowing teams to calculate recall and false-positive rates and continuously tune parameters. But if traffic never passes through the classifier at all, monitoring dashboards may still appear completely normal, because the system is measuring only “requests that entered the inspection pipeline,” not “all requests that should have been inspected.”

What was truly missing was coverage validation.

Why Did It Take Nearly a Year to Discover?

Anthropic has not yet fully explained in the currently available public information why the gap was able to persist from May 2025 to April 2026. However, judging from the typical architecture of large model platforms, the issue was likely not simply a filter process going down, but a governance breakdown involving personnel, vendors, and traffic routing.

External contractors generally do not directly use public-facing product interfaces. They may access models through vendor-built platforms, dedicated labeling tools, evaluation sandboxes, or separate API gateways. These environments must meet requirements for data isolation, task distribution, and quality review, so they often have separate identity systems and invocation paths.

Separate paths are not inherently a problem. The question is whether safety policies are inherited by default.

If production user requests are required to pass through a classifier, while a contractor platform requires a team to manually enable a switch, then a migration, model version update, or vendor integration could disable the defense. Worse still, contractors, model teams, security teams, and vendors may each see only partial metrics, with no one responsible for end-to-end coverage.

This incident exposes at least four questions that deserve scrutiny:

• Why were high-risk classifiers not mandatory and enabled by default across all entry points?
• Why did the system not issue an alert for model calls that produced no classification result?
• Why were external vendor personnel able to access highly capable models despite inadequate screening?
• Why did a traffic coverage gap involving 133 million conversations take nearly a year to identify and disclose?

For AI labs, having an advanced classifier is only the first step. Proving that every piece of traffic requiring protection actually passed through it is an even more fundamental engineering challenge.

An Awkward Gap in the ASL-3 Defenses

In May 2025, Anthropic activated AI Safety Level 3, or ASL-3 protections, when it released Claude Opus 4. That timing broadly coincides with the beginning of this gap.

The core judgment behind ASL-3 is that once frontier models reach a certain level of capability in fields such as chemistry and biology, they may not enable someone with no background to directly build a weapon, but they could significantly lower the operational barriers for people with relevant expertise. Platforms therefore need deployment restrictions, access controls, and continuous monitoring in addition to controls on model capabilities.

The logic itself is sound.

Dangerous knowledge is not a simple set of keywords. Ordinary researchers may ask about viral vectors, protein design, or experimental conditions, while malicious users can break their goals into dozens of seemingly harmless questions. Relying solely on model refusals can leave systems vulnerable to multi-turn conversations, role-playing, encoded obfuscation, and task decomposition. Relying solely on keyword filters, meanwhile, can block large amounts of legitimate research.

By using real-time input and output classifiers, Anthropic was effectively adding another dynamic security checkpoint outside the model. But this incident shows that writing a safety level into policy does not mean it has been implemented in every environment. A lab can publish a rigorous risk framework while its actual systems still develop gaps because of exempt accounts, vendor channels, or configuration drift.

This is not a problem unique to Anthropic. It reflects a new phase the entire industry is entering: the risks facing leading model companies no longer arise only from whether a model “can do something dangerous,” but also from whether a complex organization can reliably deploy safety policies across dozens of teams, vendors, and product entry points.

Stricter Classifiers Can Also Block Legitimate Research

Biosafety classifiers are not necessarily better simply because they are stricter.

Anthropic has previously acknowledged that overly aggressive blocking can interfere with legitimate research. Many specialized questions in biology, drug development, and public health may closely resemble dangerous requests in their vocabulary and experimental procedures. If a classifier considers only the topic without assessing user intent, the actionability of the information, and the context, it will lock out researchers and malicious users alike.

According to public materials, Anthropic has at times routed certain biology- and chemistry-related requests to models with more mature capability boundaries and safety policies as part of its safety arrangements for later models, with plans to gradually narrow the scope of restrictions as classifiers improve.

This reflects the dilemma currently facing frontier model platforms:

• If the threshold is too low, disguised high-risk requests will slip through;
• If the threshold is too high, legitimate research, teaching, and compliance analysis become difficult;
• Relying entirely on account identity cannot address account sharing or insider risk;
• Relying entirely on content-based judgments makes it difficult to understand the actual usage scenario.

A more mature approach should therefore go beyond a binary “allow or block” decision and instead combine user identity, historical behavior, request frequency, domain credentials, model capabilities, and the actionability of outputs to apply tiered controls. For example, low-risk content could receive a normal response; medium-risk requests could receive less detail or be routed for human review; and high-risk requests could be blocked outright and referred for a security investigation.

No matter how sophisticated the policy is, however, requests must first enter the safety system. This incident failed at precisely that first layer.

“No Abuse Found” Still Requires Cautious Interpretation

Anthropic’s investigation concluded that it has so far found no evidence that these requests were actually used for abuse. That conclusion is important, but so are its limitations.

First, a lack of evidence does not mean risk can be ruled out. Because the safety classifier was not running for an extended period, the platform lacked some of the real-time signals that would normally have been used to flag suspicious behavior. A retrospective investigation can only reanalyze stored conversation logs, account information, and external clues. Its effectiveness depends on whether the logs are complete, whether the detection rules are sufficient, and whether contractor identities can be reliably traced.

Second, there is also the question of how abuse is defined. Directly asking how to build a biological weapon is clearly high-risk behavior, but actions such as probing classifier boundaries, gathering experimental information step by step, or exporting results from a labeling platform may not be easy to identify from a single conversation.

Finally, a pool of approximately 50,000 people itself expands the insider-risk surface. Inadequate vetting by external vendors does not mean these workers acted maliciously, but under the principle of least privilege, a larger and more varied workforce requires stronger mandatory monitoring and more granular access controls.

Anthropic therefore needs to provide more than a promise to “tighten contractor requirements.” It should provide verifiable remediation results, including:

• Whether all historical conversations have been rescanned;
• Whether high-risk requests or samples that the classifier should have blocked were found;
• Whether automated reconciliation has been established between global traffic volumes and classifier invocation volumes;
• Whether vendors have been stripped of the authority to decide safety policies themselves;
• Whether an independent team has verified the remediation, rather than relying solely on self-review by the original business team;
• Whether safety controls now have non-bypassable, fail-closed defaults.

Without answers to these questions, the remediation may remain procedural rather than technical.

A Lesson for Developers: Test Coverage, Not Just Accuracy

This incident also has practical implications for enterprises deploying large models. Many teams treat safety engineering as integrating a content moderation model and then monitoring its accuracy and latency. But production incidents often occur outside the classifier: a new entry point is not integrated, streaming output is checked only at the beginning, batch-processing jobs bypass the gateway, or internal test accounts receive permanent exemptions.

At a minimum, developers should establish three categories of metrics:

1. Coverage metrics: Whether total model invocations can be matched one-to-one with safety classifier invocations;
2. Execution-result metrics: Whether every check returns a clear result, and whether timeouts and errors trigger blocking by default;
3. Exception-audit metrics: Which accounts, environments, and tasks can bypass checks, when those exceptions expire, and who approved them.

The most important consideration is the failure policy. In high-risk environments, a classifier timeout should not automatically allow a request through, nor should calls to highly capable models continue when safety configurations are missing. “Fail closed” means locking the door when the access-control system breaks, rather than allowing everyone in by default.

Internal evaluation and red-team environments should not automatically be treated as trusted zones either. To find model boundaries, testers often submit more dangerous requests than ordinary users do. If an evaluation sandbox is still connected to the real internet, external tools, or production data, its risk may be even higher than that of the public product.

This Was Not a Filter Incident, but an AI Safety Delivery Failure

Anthropic’s voluntary disclosure at least shows a willingness to subject its internal safety failures to public scrutiny, which is more valuable than hiding the incident within its vendor ecosystem. But given the scale and duration of the incident, transparency alone cannot offset the governance gap.

The 133 million conversations do not mean that large-scale biological-risk abuse occurred, but they are enough to prove one fact: the safety capabilities of frontier model companies cannot be judged solely by model cards, red-team results, and policy documents. We must also examine whether those mechanisms truly cover every production and evaluation pipeline.

As models grow more powerful, the safety question is shifting from “what will the model answer?” to “who can access it, through which path, and whether the surrounding controls remain effective at all times.” The former is a model research problem; the latter is more akin to cloud security, supply-chain management, and reliability engineering.

Anthropic stumbled over the latter.

For the industry, the real danger is not that a classifier occasionally makes the wrong judgment, but that the system continues to behave as though everything is normal when the classifier is not running at all.

References

• ITHome: Anthropic Discloses Safety Vulnerability; Bioweapon Filter Was Inoperative for Nearly a Year — Summarizes the time span, number of contractors, conversation volume, and subsequent investigation findings disclosed in Anthropic’s safety report.
• Zhihu: Discussion of Anthropic’s Report — Provides additional interpretation of the report’s disclosure that contractor traffic did not pass through the biological-risk blocking classifier.