Don’t Let Coding Agents Copy Everything They See

Recently, a developer shared their experience modifying an LLM coding agent. What originally appeared to be a complete “goal → decomposition → research → specification → implementation” pipeline revealed a typical problem in practice: whatever the agent finds, it wants to implement.

If five papers present five methods, the model may not choose among them. Instead, it may combine all five into a single implementation. If an interface has several optional inputs, it may accept them all. If the research materials mention an abstraction layer, it may create that as well. The end result is usually not code that fails to run, but a system far more complex than the requirements demand.

This problem deserves attention because it exposes a fundamental weakness in current coding agents: LLMs are good at expanding the space of possibilities, but they are not naturally good at freezing engineering decisions.

The solution is not to add another instruction such as “please keep it simple,” but to place genuine decision and specification gates between research and implementation. Research can provide candidates, but it cannot directly become a construction blueprint.

Why Does a Seemingly Complete Pipeline Still Run Out of Control?

The developer’s original process was not careless:

Goal
  -> Decompose
  -> Research
  -> Specification
  -> Implementation

The agent first understands the goal, decomposes it into subproblems, researches each subproblem, produces a detailed specification, and finally writes the code. Judging by the stage names alone, this is already far more rigorous than prompt-driven “vibe coding.”

The problem is the lack of semantic isolation between stages.

Research typically produces content such as:

• Proven mainstream approaches;
• Inspiring alternatives that are unsuitable for the current project;
• Experimental designs from academic papers;
• Mechanisms added to handle unusual scales or edge cases;
• Information that is not directly relevant to the current requirements but is highly related semantically.

When human engineers review these materials, they naturally distinguish between “knowing that this option exists” and “deciding to adopt this option for the project.” Models, however, can easily interpret relevance as necessity: if something appears repeatedly in the context, it should be included in the specification; if it is included in the specification, it should be implemented in code.

As a result, research findings leak through the context window all the way into the implementation stage.

For example, suppose the task is merely to implement a retrieval-ranking strategy for a small- to medium-scale dataset. After research, the agent discovers five approaches: rule-based ranking, vector retrieval, reranking models, graph-based propagation, and online learning. Without an explicit decision layer, it may design a pipeline compatible with all five, complete with a unified interface, strategy factories, dynamic configuration, fallback logic, and multiple test suites.

The code may look highly “professional,” but the engineering objective only requires one of those approaches.

This is a particularly subtle failure mode in agentic coding: the agent does not deliver the wrong answer; it delivers too much of an answer. The compiler will not complain, and all unit tests may even pass. The real problem only emerges during maintenance, deployment, or requirement changes.

Why LLMs Prefer to “Implement Everything”

This is not simply a matter of the model being lazy. Quite the opposite: the model is often overly proactive.

1. Generative Models Find It Easier to Take the Union Than to Make Exclusive Decisions

LLMs predict what comes next based on context. Once multiple methods have been included in the context, organizing them into a “comprehensive solution” is generally more natural than explicitly rejecting four of them.

In natural-language responses, broad coverage often indicates quality. In engineering implementations, however, broad coverage can mean a larger attack surface, state space, and maintenance burden.

2. Research Evidence and Requirement Constraints Are Not Layered

If paper abstracts, user goals, repository code, and architectural constraints are all inserted as ordinary text into the same prompt, the model sees them merely as tokens at different positions. They have no strict hierarchy of authority.

To a human, “the paper proposes method B” is entirely different from “this project must use method B.” But for an agent without structured constraints, the semantics of the two can easily become entangled.

3. Agent Incentives Usually Favor Visible Output

Many coding-agent loops work like this: read the task, modify files, run tests, and continue fixing issues. Tool calls and code diffs are observable signs of progress. By contrast, stopping implementation, requesting a decision, or explicitly documenting what will not be done can look like a failure to complete the task.

As long as the acceptance mechanism primarily checks whether the result runs, the model has an incentive to keep adding code rather than narrowing the scope.

4. “Minimum Implementation” Has Not Been Converted into Verifiable Criteria

“Keep it simple” and “do not overengineer” are merely preferences, not constraints. Such adjectives alone do not tell the model how many input parameters are allowed, whether dependencies may be added, whether new abstractions may be created, or how many algorithms need to be supported.

Without an explicit complexity budget, simplicity is left to the agent’s interpretation.

What Is Really Missing Is a Decision Stage

The original pipeline should include at least one explicit Decision Gate between Research and Specification:

Goal
  -> Decompose
  -> Research
  -> Decision Gate
  -> Specification Gate
  -> Implementation
  -> Verification

The research stage answers, “What approaches are feasible?” The decision stage answers, “Which one are we choosing this time, and why?” The specification stage answers, “How exactly will the selected approach be implemented?”

These three cannot be merged into a single long document.

A practical decision artifact can use a structured format:

decision_id: ranking-001
problem: Provide offline retrieval ranking for 100,000 documents
selected_option: vector_retrieval
reason:
  - Vector database infrastructure is already available
  - Latency target is below 200 ms
  - Online feedback learning is not currently required
rejected_options:
  graph_ranking: Deployment and data maintenance costs are too high
  online_learning: No stable feedback signal is available
  cross_encoder_only: Full computation cannot meet latency requirements
non_goals:
  - Do not implement dynamic switching between multiple algorithms
  - Do not support a training pipeline
  - Do not introduce a new strategy abstraction layer
open_questions:
  - Should the default top_k value be configurable by the business team?
status: approved

The most important fields here are not selected_option, but rejected_options and non_goals.

During human design reviews, teams usually discuss why another option was not selected. Agent workflows, however, often preserve only the final answer, allowing rejected methods to “come back to life” in later stages. Explicitly recording the reasons for rejection creates a negative boundary for the model.

A Specification Gate Is Not Just Another Generated Design Document

Once a decision has been approved, the specification must still be checked for hidden scope expansion.

An executable specification should answer at least the following questions:

• Which modules and files may be modified?
• What are the input, output, and error semantics?
• Which dependencies may be added?
• Which behaviors must remain backward-compatible?
• Which candidate approaches have been excluded?
• Which scenarios are covered by acceptance tests?
• Under what conditions must the agent stop?

For example, instead of writing “implement a flexible and extensible retrieval module,” provide verifiable constraints:

implementation_scope:
  editable_paths:
    - src/search/vector_ranker.py
    - tests/search/test_vector_ranker.py
  max_new_dependencies: 0
  max_new_public_interfaces: 1
  permitted_algorithm:
    - cosine_similarity
  forbidden_features:
    - plugin_system
    - runtime_algorithm_selection
    - online_training
acceptance_tests:
  - An empty query returns an empty list
  - No error is raised when top_k exceeds the number of candidates
  - Benchmark for 100,000 vectors completes in under 200 ms
stop_condition:
  - All acceptance tests pass
  - No files outside the approved scope have been modified

Here, “do not overengineer” has been translated into budgets for paths, dependencies, interfaces, and features. The model no longer has to guess what “simple” means.

This is why research and specification gates are genuinely useful: they do not merely make the model think for a few more rounds. They compile engineering intent into boundaries that machines can verify.

Gates Must Be Able to Block Execution, or They Are Merely Ceremonial

Many teams already ask agents to write a plan before writing code, but the practical effect is limited. The reason is that the so-called plan is merely a paragraph in the chat history. Once the agent finishes writing it, it immediately begins modifying the repository, with no mechanism to verify whether the plan is acceptable.

A real gate should be a state machine, not a polite phrase in a prompt:

if state == "RESEARCHED":
    require(decision.selected_option)
    require(decision.rejected_options)
    require(decision.non_goals)
    state = "AWAITING_DECISION_APPROVAL"

if state == "SPECIFIED":
    require(spec.editable_paths)
    require(spec.acceptance_tests)
    require(spec.stop_condition)
    state = "AWAITING_SPEC_APPROVAL"

if state != "APPROVED_FOR_IMPLEMENTATION":
    deny_write_tools()

The key is not these few lines of pseudocode, but the final deny_write_tools(): until the decision or specification has been approved, the agent must not receive access to high-impact tools such as file writing, dependency installation, or database migration.

Permission controls are more reliable than prompts. If you tell an agent “do not write code yet,” it may still append an implementation in the second half of its response. Withholding write tools entirely creates a system-level constraint.

The gate reviewer can be a human or another model, but each is appropriate for different scenarios:

• Decisions involving product trade-offs, data models, external interfaces, or migration risks should be approved by a human;
• Checks for whether a specification includes acceptance criteria or references unselected approaches can be delegated to an independent model;
• Small, low-risk tasks may be approved automatically, but should retain a complexity budget;
• For production write operations, permission systems, and payment logic, the same agent should not both propose and approve its own changes.

Having one model generate a proposal and then review itself within the same context can easily become an exercise in self-confirmation. A safer approach is for the reviewer to read only the goal, decision record, and specification—not the implementation agent’s full reasoning process—and require it to return an explicit approval or rejection with reasons.

Research Materials Also Need “Purpose Labels”

In addition to adding a decision stage, research artifacts should themselves be categorized. A simple classification system is:

1. Constraint: A fact or restriction that must be followed, such as protocol specifications, existing interfaces, and latency targets;
2. Candidate: An optional implementation approach that must pass through a decision process before entering the specification;
3. Context: Information that helps explain the problem but must not directly generate requirements;
4. Rejected: An approach that has already been evaluated and excluded;
5. Unknown: Something for which the evidence is insufficient and that requires experimentation or human confirmation.

This prevents the automatic conversion of “information” into “requirements.” A method mentioned in a paper should be classified as a Candidate by default, not promoted directly to a Constraint.

When retrieval-augmented generation is used, each conclusion should also preserve its source and applicability conditions. For example, an algorithm that performs well on an offline dataset with millions of records is not necessarily suitable for a low-latency online service. The agent should record not only the conclusion, but also its boundaries.

Not Every Task Warrants the Full Process

Research and specification gates increase latency and token costs. Running a five-stage review just to correct a variable name in an internal script would clearly be process overkill.

A more reasonable approach is to tier the process by risk:

| Task Type | Recommended Process | | --- | --- | | Copy, comments, or small-scale test fixes | Implement directly and run tests | | Single-module feature with no external interface changes | Simplified specification gate | | New dependency, new database table, or public API change | Both decision and specification gates | | Authentication, payments, data deletion, or production migration | Human approval + permission isolation + rollback plan |

Triggers can also be defined. For example, the approval level can be escalated automatically when research identifies more than two candidate methods, the agent plans to add a dependency, the scope of modifications exceeds expectations, or the specification contains phrases such as “extensible framework” or “general-purpose plugin system.”

In other words, gates should also operate according to risk. They must not become another bureaucratic template.

What This Means for Agentic Programming

Over the past year, industry discussions have often focused on how much code models can write, how long they can work continuously, and whether they can run tests and submit pull requests. But once coding agents enter large repositories, the real constraint is often not their ability to generate code, but their ability to control scope.

Knowing how to write code only solves the question of “how to implement it.” Reliable software engineering must continuously answer several other questions:

• Why should it be implemented?
• Why was this approach selected?
• What is explicitly out of scope?
• Who is allowed to change an approved decision?
• At what point should the work stop?

The developer’s approach does not propose a complicated new framework, but it captures a critical point: research should guide implementation, but it must not automatically become implementation.

Our assessment is that gates like these are more valuable than merely extending an agent’s runtime. Stronger models may reduce syntax errors and local logic errors, but they will not necessarily reduce overengineering automatically. In fact, the more capable the model and the broader its tool permissions, the faster it can propagate a mistaken assumption across an entire codebase.

A mature agent-development environment of the future will probably be more than a chat window capable of operating a terminal. It will look more like a permission-constrained engineering state machine: a researcher expands the candidate space, an architecture role makes trade-offs, a specification reviewer freezes the boundaries, an implementation agent works only within the approved scope, and a verifier finally checks whether the result has crossed those boundaries.

Developers’ work will not disappear. It will shift from producing code line by line to defining constraints, approving trade-offs, and designing feedback loops.

For teams, there are three minimal changes that can be implemented today: store research conclusions and engineering decisions in separate files; add non_goals to specifications; and disable write tools until the specification has been approved. These changes may not be as eye-catching as “automatically completing an entire project,” but they are much closer to the true starting point for controllable coding agents.

References

• My LLM kept implementing every method it found, so I added research and specification gates: A developer shares how an LLM agent overimplemented the approaches it discovered during research, and how research, decision, and specification gates were introduced.
• Discussion of LLM agents that can execute code: A community discussion about models invoking execution tools such as code interpreters and terminals, useful for understanding the relationship between tool permissions and agent capabilities.
• Analysis of OpenAI’s million-line coding-agent practices: A Chinese-language analysis of constraints, feedback loops, documentation structures, and dependency rules in large-scale agentic coding.