Huawei Adds a Human Gatekeeper to AI-Powered Vulnerability Hunting

AI Can Hunt for Vulnerabilities, but It Cannot Shift the Cost of Verification onto Vendors

On October 14, the Huawei Security Response Center announced updated vulnerability report handling rules for its Device Security Reward Program. The rules explicitly require that any vulnerabilities discovered with AI assistance or automation must undergo manual analysis and verification before submission, along with evidence demonstrating actual security impact.

This does not mean Huawei is rejecting AI-powered vulnerability discovery. Quite the opposite: AI has already entered real-world vulnerability discovery workflows. The problem is that models have reduced the cost of submitting reports to nearly zero, while the cost vendors incur to verify those reports has not fallen accordingly.

Since the beginning of this year, Huawei has received a large number of vulnerability reports produced with the assistance of AI tools. According to the company, most of these reports included a proof of concept (PoC), but many were still misled by AI “hallucinations”: the code looked plausible, and the vulnerability descriptions used the right security terminology, but when deployed against the target version and triggered through an actual entry point, the issues could not be reproduced at all—or they merely demonstrated abnormal program behavior without proving that a security boundary had been breached.

Huawei has now drawn the line very clearly: AI may assist in discovery, analysis, and test-case generation, but a human must take responsibility for the final report.

Having a PoC Does Not Necessarily Mean There Is a Real Vulnerability

The most noteworthy part of this rules update is not the phrase “manual verification,” but Huawei’s new emphasis on a complete chain of evidence from a real attack entry point to demonstrable security impact.

In traditional vulnerability reports, a PoC is generally used to prove that a flaw can be triggered. Since the rise of generative AI, however, the evidentiary value of PoCs has rapidly diverged.

One type of PoC is verification code that a researcher has successfully run on a real device, against a real software version, through an accessible interface. Another type is merely a “plausible guess” assembled by a model from function names, historical CVEs, code snippets, and common vulnerability templates. The latter may pass syntax checks and may even generate screenshots showing the expected output, but it does not necessarily correspond to a real attack surface.

Common false positives include:

• Calling interfaces that have already been removed, disabled, or made unreachable in the current version;
• Analyzing only an incomplete code snippet without considering permission checks elsewhere in the call chain;
• Treating debug modes, internal test environments, or entry points requiring system privileges as remotely accessible attack surfaces;
• Finding a crash and immediately describing it as remote code execution;
• Presenting quality defects, abnormal behavior, or configuration issues as security vulnerabilities;
• Inferring an out-of-bounds access, injection flaw, or authorization bypass from static code without reproducing it on the latest system version;
• Generating a formally complete exploit whose prerequisites do not exist on real devices.

Vulnerability reward programs have never been about whether “this code looks suspicious.” They assess whether an attacker can compromise the confidentiality, integrity, or availability of a system under real-world constraints.

Huawei therefore requires reports to include a complete, dynamically executable PoC and encourages the submission of a complete exploit chain. Reports must also clearly specify the affected product, system version, impact, and reproduction steps, and include complete logs, screenshots of results, or demonstration videos. More importantly, the attack path must begin from an entry point available in a real attack scenario and prove that it ultimately causes genuine security impact.

In other words, a report cannot stop at “there may be a problem here.” It must answer three questions:

1. Where does the attacker enter?
2. Can the attack chain be completed on the currently available version under its default security boundaries?
3. What capability does the attacker actually gain upon success, or which security property is compromised?

This effectively shifts the evaluation standard from “the model thinks it looks like a vulnerability” back to “an attacker can actually exploit it.”

The Latest HarmonyOS Version Becomes a Hard Requirement

The new rules also stipulate that the product covered by a report must fall within the scope of the reward program and must have been upgraded to the latest HarmonyOS version.

This requirement will immediately eliminate a batch of reports based on historical code and legacy interfaces. Large models are particularly good at identifying similar patterns in publicly disclosed vulnerabilities, old SDK documentation, historical repository commits, and existing research articles. However, they may not correctly determine whether a given function still exists in a production release, nor whether patches, permission models, and system architecture have changed.

An issue that appears only in old firmware, an outdated development build, or a disabled interface may still have research value, but it may not meet the acceptance criteria of the current Device Security Reward Program.

Huawei’s emphasis on “the latest callable interfaces” also forces automated tools to improve their environment-awareness capabilities. An AI vulnerability discovery system suitable for production use cannot merely read code and generate hypotheses. It must also be capable of identifying versions, deploying to devices, preparing system state, invoking interfaces, collecting results, and repeatedly validating findings.

This part of the process is the hardest to automate—and the best way to distinguish whether a vulnerability discovery agent is a genuine research tool or merely a text-generation machine for mass-producing reports.

Three Warnings, and Five Invalid Reports May Get an Account Banned

The other side of the rules is the enforcement mechanism.

Huawei’s list of noncompliant behavior includes continuously, repeatedly, or extensively submitting reports generated directly by AI without manual reproduction; failing to trigger an issue through the latest callable interface; analyzing only incomplete code snippets; or failing to provide verification results sufficient to demonstrate actual security impact.

Such reports will be rejected outright, and the platform will not explain the reason for every individual rejection:

| Cumulative record | Action | | --- | --- | | Submission of an invalid report | Rejected outright, with no individual explanation | | A total of 3 invalid reports | The platform warns the account holder | | A total of 5 invalid reports | The platform may ban the account from the Security Reward Program |

If participants distort facts, maliciously disparage the program, abuse, threaten, or harass staff, or flood the platform with messages, Huawei may immediately terminate the eligibility of the account holder and their team or organization, or impose a permanent ban.

“Five invalid reports may result in an account ban” may sound strict, but it is not excessive in the context of AI-powered mass generation. Security review is not ordinary content moderation. Analyzing a potentially critical vulnerability often requires obtaining the relevant software version, setting up an environment, confirming interfaces, reviewing code, collecting logs, and sometimes coordinating with product teams. Even if the report ultimately proves to be a false positive, the review costs have already been incurred.

At the heart of the problem is a clear asymmetry: a submitter can use a model to generate dozens of reports in minutes, while a vendor’s engineers may need hours to investigate each one. Without cost constraints, reward programs will be flooded with low-quality content, forcing genuinely valuable reports to wait longer in the queue.

Rather than banning AI, Huawei is using account reputation and submission thresholds to shift part of the verification burden back to report submitters. This is more practical than simply trying to determine whether a report was “written by AI,” because the origin of text is nearly impossible to identify reliably, while the reproducibility of evidence can be evaluated.

Scope for Tool-Related Vulnerabilities Also Narrows, with DevEco Reports No Longer Generally Accepted

In addition to addressing AI-generated false positives, Huawei has adjusted the scope of accepted vulnerabilities affecting device-related tools.

Starting September 30, 2026, the Huawei Device Security Reward Program will temporarily stop accepting ordinary vulnerabilities in device tools such as DevEco. Two main categories of exceptions will remain eligible for submission:

• Vulnerabilities that could enable supply-chain poisoning of public repositories for Huawei’s internally developed tools;
• Vulnerabilities that could use the affected tools as a stepping stone to compromise the security of Huawei devices.

These reports will be evaluated comprehensively based on their specific impact.

This change means vulnerabilities that only affect the local operation of development tools, without extending into the software supply chain or reaching end devices, will receive significantly lower priority. Huawei is concentrating its limited review resources on two high-value paths: supply-chain risks affecting large numbers of developers, and attack chains that can ultimately reach real devices.

From a defensive perspective, this is a narrowing of scope. For researchers, it means that the criteria for choosing research targets have changed. In the past, proving that a DevEco component contained a flaw might have been enough for a report to enter the review process. Now, researchers must also answer additional questions: Can the flaw poison a public repository, affect build artifacts, or further threaten end devices?

Huawei Is Not Alone: AI-Generated Junk Reports Are Draining the Entire Industry

Huawei’s latest changes are not an isolated event.

As of October 1, 2026, Google has paused acceptance of product vulnerabilities under its Open Source Software Vulnerability Rewards Program, OSS VRP, and plans to announce restructuring progress in the first quarter of 2027. According to reports, Google engineers and open-source maintainers had previously received thousands of low-quality reports, many of which claimed to identify serious vulnerabilities but ultimately proved to be erroneous AI-generated conclusions.

The two companies have adopted different levels of enforcement: Google has paused parts of its program outright, while Huawei continues to accept AI-assisted reports but requires manual reproduction and has introduced a banning mechanism for repeated invalid submissions.

Huawei’s approach is relatively moderate and more closely resembles a governance model that can operate over the long term. Completely rejecting AI-generated content is unrealistic because AI is already integrated into static analysis, fuzzing, taint analysis, patch comparison, and exploit development. It is becoming increasingly difficult to clearly distinguish whether a particular explanation, test script, or even attack-chain analysis in a report was “written by a human” or “written by a machine.”

What should be reviewed is the result, not the identity of the tool.

A report completed with AI assistance but fully verified by a researcher is no less valuable than a purely manual report. Conversely, a report written entirely by a human but lacking reproducibility and actual impact should likewise not receive a reward.

Rankings released in June of this year also demonstrate that AI-powered vulnerability discovery is not an inherently flawed proposition. A vulnerability discovery agent developed by 360 previously won first prize in the Huawei Device Security Reward Program and ranked near the top of the annual leaderboard with a high score. It represents a different approach: an automated system that not only generates vulnerability hypotheses, but also consistently produces results that vendors can verify and ultimately bring into the remediation cycle.

This rules update therefore does not mean that “AI vulnerability discovery has failed.” Instead, it reflects the industry’s growing distinction between two fundamentally different capabilities: identifying something suspicious and proving that it is a vulnerability.

The former is rapidly becoming commoditized. The latter remains expensive.

Security Researchers Need to Redesign Their Workflows

For researchers using large language models or agents to assist with vulnerability discovery, the new rules mean that vendor reward platforms can no longer be treated as free verification services. After a model produces its output, researchers must at minimum perform another round of internal screening against a real environment.

A more reliable submission process should include:

1. Pin Down the Target Version

Clearly specify the device model, HarmonyOS version, patch level, and application or component version. Avoid directly mapping issues found in old code repositories onto current production releases.

2. Verify That the Entry Point Is Reachable

Confirm whether an ordinary attacker can access the target interface and whether doing so requires a system signature, debugging privileges, physical access, an unlocked device, or prior control of a privileged process. All prerequisites must be truthfully documented in the report.

3. Reproduce the Issue Multiple Times

Repeat the trigger in a clean environment to rule out false appearances caused by caches, residual state, test configurations, or one-off crashes. Triggering an issue once does not mean the vulnerability exists reliably.

4. Distinguish Abnormal Behavior from Security Impact

Application crashes, service restarts, and log errors do not inherently constitute critical vulnerabilities. A report must prove whether the issue results in privilege escalation, data leakage, authentication bypass, arbitrary code execution, persistent denial of service, or escape from a security boundary.

5. Preserve Complete Evidence

Retain original requests, device logs, crash stacks, version information, screenshots, and demonstration videos. Do not submit only a model-generated summary, because it may omit critical context that determines whether the vulnerability is valid.

6. Have a Human Review the Model’s Conclusions

Pay particular attention to whether the model has fabricated APIs, misinterpreted permission semantics, exaggerated the severity of a vulnerability, or presented a theoretical attack path as a verified fact. Every key conclusion in the report should be supported by experimental results.

This will reduce the number of submissions but increase their validity rate. For teams that genuinely want to participate in reward programs over the long term, the validity rate matters more than the total number of reports. Account reputation, vendor response times, and future working relationships will all ultimately be affected by the quality of past submissions.

This Human Gate Is Necessary, but Not Sufficient

Huawei’s new rules address the most urgent issue today: low-cost, automatically generated vulnerability conjectures cannot be allowed to consume unlimited amounts of expensive manual review capacity. Requirements for real entry points, the latest versions, complete PoCs, and actual security impact are also more enforceable than a vague instruction not to “submit AI reports.”

However, the rules could still be refined further.

For example, the platform could eventually provide more standardized rejection labels, structuring reasons such as “not reproducible,” “version mismatch,” “entry point unreachable,” “no security impact,” and “out of scope.” Huawei currently states that invalid reports will be rejected outright without an individual explanation. While this saves review resources, it may also make it difficult for serious but inexperienced submitters to improve.

A more ideal approach would be for the platform to provide a machine-readable submission template that requires submitters to specify the target version, attack prerequisites, reproduction success rate, actual impact, and evidence hashes. An automated environment could then perform basic deduplication and executability checks before human review. This would avoid requiring engineers to write lengthy responses to every junk report while still allowing qualified researchers to understand what went wrong.

The evaluation metrics for AI vulnerability discovery products should also change. The number of repositories scanned and the number of findings generated are no longer meaningful core metrics. More valuable indicators include:

• Reproduction rates on the latest production versions;
• The proportion of valid vulnerabilities confirmed by vendors;
• The completion rate of attack chains from entry point to impact;
• The ability to filter duplicate reports and previously disclosed vulnerabilities;
• Understanding of permissions, versions, and runtime environments;
• The number of findings that ultimately enter the patching and remediation cycle.

The security industry does not lack “text that looks like a vulnerability.” What it lacks are conclusions that withstand reproduction.

The human gate Huawei has now added to AI-powered vulnerability discovery will make life harder for mass submitters in the short term and may reduce the total number of reports the platform receives. But from the perspective of the reward program’s long-term quality, this is necessary noise reduction. AI should help researchers expand the search space, not package unverified guesses and dump them on vendors.

When generating reports becomes cheap, proving that they are true becomes the new scarce capability.

References

• AI-Discovered Vulnerability Reports Contain False Positives; Huawei Device Security Reward Program Requires Manual Analysis and Verification Before Submission — Introduces the Huawei Device Security Reward Program’s new reporting requirements, enforcement measures, and changes to the scope of accepted tool-related vulnerabilities.
• Flood of AI “Hallucination” Reports Overwhelms Maintenance Teams; Google Pauses Part of Its Open-Source Vulnerability Rewards Program — Provides industry context on changes to Google’s OSS VRP and the impact of low-quality AI-generated vulnerability reports on review resources.