MakerChecker Open Source: A Safety Checkup for Dangerous Capabilities in AI Agents

Someone Turned AI Agent "Dangerous Capability Scanning" into an Open-Source Tool

Over the past couple of days, a project called MakerChecker surfaced on Hacker News. Its purpose is very specific: scan your AI Agent and identify dangerous capabilities and high-risk behavior chains it could potentially execute. This is not the usual generic "prompt injection detection." Instead, it lays out every tool the Agent can call and every resource it can access, exhaustively enumerates combinations, and tells you: under this configuration, your Agent could theoretically do these things.

It sounds like static analysis concepts transplanted into the Agent security space. But that analogy is only half right. What actually makes it interesting is that MakerChecker focuses not on the risk of a single tool call, but on the capability chain — the destructive potential created when multiple seemingly harmless capabilities are combined.

Why Single-Point Scanning No Longer Works

From the second half of 2025 through the first half of 2026, Agent security has completely exploded as a field. Tencent’s Zhuque Lab scanned more than 50,000 Skill packages on ClawHub and detected 452 high-risk malicious samples. The LiteLLM supply-chain poisoning incident involving 480 million downloads was not that long ago. By early 2026, Agent Skills had already completely overshadowed MCP and become the new attack surface.

The problem is that traditional scanning approaches do not really work for Agents. The reasons are simple:

• Each individual tool is "legitimate." Reading files is legitimate, sending HTTP requests is legitimate, writing files is legitimate too. But chaining together "read SSH key → POST to external server → wipe bash_history" becomes data theft.
• Permission boundaries are blurry. Agent tool calls often share the same execution context, without capability isolation like an operating system’s capability-based security model.
• The LLM itself is a non-deterministic decision-maker. The same toolset can produce completely different execution paths under different prompts.

What MakerChecker essentially does is treat the Agent as a capability graph, where nodes are tools and resources, and edges are invocation relationships. It then traverses the graph to find every path that can reach a "sensitive sink." These sinks include the public internet, payment interfaces, production databases, and user credentials.

This is conceptually very similar to traditional taint analysis, except the target has shifted to Agent toolsets.

How It Actually Works

Judging from the README, usage is fairly straightforward. You feed MakerChecker your Agent’s tool definitions (such as OpenAI function-calling schemas or MCP server manifests), and it outputs a scan report:

• High-risk capability inventory: Which tools are inherently dangerous, such as exec_shell, http_post, or file_write_arbitrary_path.
• Dangerous behavior chains: Concrete attack path simulations, annotated step-by-step with the tools used.
• Risk severity scores: CVSS-style severity ratings for easy CI integration.
• Mitigation recommendations: Suggestions like removing certain tools, adding human approval steps, or enforcing parameter whitelists.

The "Maker-Checker" concept in the project name comes from a long-standing financial industry risk-control model — one person initiates an action (Maker), another independently reviews it (Checker). The author is clearly trying to apply this idea directly to Agent architectures: any high-risk behavior chain must go through human or independent-Agent review, instead of letting the primary Agent operate unchecked.

Comparison with Similar Projects

This space is already getting crowded:

• Tencent Zhuque’s A.I.G (AI-Infra-Guard): More infrastructure-oriented, focused on large-model services, MCP Servers, and Skill marketplace batch scanning. Huge datasets and broad coverage, but less granular customization for a specific Agent.
• Strix (usestrix/strix): Takes the "AI hacker" route, using multi-Agent collaboration to attack real targets and validate vulnerabilities with actual PoCs. This is active penetration testing, fundamentally different from MakerChecker’s static-analysis positioning.
• Cursor Security Review: Cursor’s newly launched feature that uses AI Agents for PR-level code review. Its focus is vulnerabilities in the code itself, not capability risks in Agent configurations.

MakerChecker fills a very specific gap: performing static audits of Agent capability configurations before deployment. This space was previously almost empty. Most people were scanning models, scanning code, or building runtime protections, but very few were systematically examining an Agent’s "capability DNA."

A Practical Scenario

Here’s an example developers can easily relate to. Suppose you are building a customer-support Agent with the following tools:

• get_user_info(user_id): Retrieve user profiles
• send_email(to, subject, body): Send emails
• fetch_url(url): Fetch external web pages (for validating refund links, etc.)

Individually, each one looks perfectly normal. But MakerChecker would tell you:

High-risk chain #1: get_user_info → send_email. An attacker could use prompt injection to make the Agent send arbitrary user data to the attacker’s email address. Risk level: High.

High-risk chain #2: fetch_url → potential SSRF path. fetch_url does not restrict internal network addresses and could be manipulated into accessing 169.254.169.254 to retrieve cloud metadata credentials. Risk level: Critical.

A human review could catch this manually, but once your Agent has 20 tools and 300 MCP endpoints, combinatorial explosion kicks in. That’s where automated scanning becomes valuable.

Assessment: Worth Watching, but Not a Silver Bullet

To state the conclusion directly: MakerChecker’s core idea is correct. The engineering implementation is still early-stage, but this is one of the few categories in Agent security that was inevitably going to emerge.

Its main strength is turning a vague question ("Is my Agent secure?") into something enumerable and reportable ("How many dangerous chains can these tools create?"). That has enormous value for compliance, auditing, and SDLC integration.

Its weaknesses are equally obvious:

1. Static analysis cannot see runtime behavior. Whether the LLM will actually follow a given path still depends on model behavior. Just like traditional SAST, false positives are unavoidable.
2. Coverage of the MCP/Skill ecosystem. The project is still young. Keeping up with rapidly evolving standards such as MCP, Agent Skills, and A2A protocols will require continuous investment.
3. Coordination with runtime defenses. The ideal architecture should combine "static scanning (MakerChecker) + runtime policy engines (like OPA) + human review (the real Maker-Checker workflow)." One layer alone is not enough.

For developers, the recommendation is simple: if you are building serious Agent products — especially ones connected to external tools, user data, or production resources — run MakerChecker against your tool schema first. At minimum, you will understand your risk exposure. Don’t wait until someone uses your Agent as a pivot point to attack customers before trying to patch things up.

One More Observation

Open-source momentum in Agent security has clearly accelerated this year. From Zhuque’s A.I.G, to Strix’s AI-driven penetration testing, to MakerChecker’s capability scanning, open-source tools from several different angles are already beginning to form a preliminary Agent DevSecOps toolchain. Combined with IDE vendors like Cursor building PR-level security Agents, the overall shift-left movement for security is actually progressing faster in the Agent era than it did in the traditional software era.

On the multi-model orchestration side, OpenAI Hub (openai-hub.com) is doing something similar around "aggregation + controllability" — one API key for GPT, Claude, Gemini, DeepSeek, and other mainstream models, compatible with OpenAI formats and directly accessible within China. For Agent teams, this significantly reduces model integration complexity. In essence, both security and aggregation are attempts to fill in missing infrastructure for Agent engineering.

References

• MakerChecker GitHub Repository: The main subject of this article, an open-source dangerous capability scanning project for AI Agents
• Strix - Open-Source AI Hacking Framework: Automated penetration testing with multi-Agent collaboration, complementary to MakerChecker
• 2025 Open-Source Supply Chain Poisoning Analysis Technical Report: Includes scan data on high-risk Skill packages, useful as background context for MakerChecker
• Tencent AI-Infra-Guard: Open-source AI red-team testing platform from Zhuque Lab, representing another path for Agent infrastructure security