MCP Workflow in Practice: From Project Planning to a Runnable Deep Learning Model

Recently I came across an interesting post on r/MachineLearning: an engineer has open-sourced the MCP workflow he’s been using for several months, designed to tackle a long‑standing issue—the painful “translation gap” between a deep‑learning project’s engineering plan and the runnable code.

This isn’t another “I built a Transformer with Cursor” show‑off post. The author’s approach is more engineering‑oriented: treat the engineer’s plan as the single source of truth, have Codex (or any MCP‑compatible agent) decompose it into blocks and implement them in dependency order, using papers only as partial references—rather than letting the agent “freestyle” to “reproduce some paper.”

My first reaction after reading it was: this system actually answers the most awkward question about current AI coding agents—they can write functions, but they can’t build a system.

Why existing coding agents can’t handle deep‑learning engineering

First, the pain point. Anyone who’s done model engineering knows how brutal the details get when going from a design doc like “I’ll implement a multimodal retrieval model, ViT‑L backbone, cross‑attention to fuse text” to runnable code on GitHub:

• Which exact ViT variant, what patch size, where to get pretrained weights
• How to align Q/K/V dimensions in cross‑attention
• How to combine losses and schedule warmup steps during training
• Tokenizer vs. image transform order in the data pipeline
• Which parameters to all‑reduce in distributed training

If you throw this at Claude or GPT‑5.1, it might produce working‑looking code, but there’s an 80–90% chance it won’t run, or it will run but yield a model completely different from the one you imagined. The reason is simple: the LLM is doing implicit planning, while the engineer’s plan is explicit—they don’t line up.

Worse, the agent often gets “creative” while implementing a component—it reads a paper abstract, thinks some trick looks cool, and silently adds it. You review the code and go, “Wait, what?”

What this MCP workflow gets right

The author’s core philosophy fits in one line: the plan is the constitution, papers are footnotes, the agent is just the translator.

The process breaks down into six steps:

1. Decompose – Split the plan into implementation blocks, each representing an independently buildable and verifiable unit
2. Find papers – Retrieve research papers relevant to each block—“relevant,” not “to be reproduced”
3. Extract details – Pull implementation details from papers that support the current plan (like a normalization trick or a hyperparameter range)
4. Write specs – Generate a component spec defining input/output, dependencies, and validation criteria
5. Implement in dependency order – The key step: rather than streaming code start‑to‑finish, the agent builds block‑by‑block after topological sorting
6. Log implementation and validation – Each completed block’s details and test results are written back into the workflow context

The elegant bit is the “papers as aids only” stance. The author repeatedly emphasizes: these papers are not to define or replicate the project. They’re sanity checks—when your plan says “use contrastive loss” without specifying the temperature parameter, the agent can look it up in the CLIP paper for a reasonable default.

This philosophy is restrained and engineering‑minded. It acknowledges a fact: AI is bad at architecture decisions but good at constrained fill‑in‑the‑blank tasks.

MCP’s role here

Many people still think MCP (Model Context Protocol) just means “mounting a file‑system tool for Claude.” But this workflow shows a far more valuable purpose—turning a complex engineering process into a set of composable tool calls.

Let’s unpack the MCP servers this workflow needs:

• plan‑parser – parses the engineering plan document, outputs a structured list of blocks
• paper‑search – interfaces with arXiv/Semantic Scholar to retrieve papers by block keywords
• paper‑extract – reads PDFs and extracts sections relevant to the current block’s implementation details
• spec‑generator – merges plan blocks and paper excerpts into a component spec
• codex‑executor – calls Codex or another coding agent to implement the spec
• verification‑logger – runs tests, records results, and updates workflow state

Each server is modular and swappable. You can use Codex today and switch to Claude Code or Cursor Agent tomorrow—so long as they meet the MCP interface, the workflow itself doesn’t change.

That’s MCP’s most underrated value: it doesn’t make agents stronger; it makes them interchangeable. Once your engineering process is MCP‑ified, the underlying models become plug‑and‑play components.

A concrete example: implementing a RAG retriever

Suppose your plan says:

Component A: Dense Retriever. Use E5‑large as encoder; encode query and doc separately; retrieve by cosine similarity. Train with in‑batch negatives + hard negatives (4 per query).

This plan is clear, but at least five implementation choices remain unspecified:

• Pooling method for E5 (mean? CLS?)
• Whether query and doc share weights
• How hard negatives are mined (BM25? previous model?)
• Training batch size and learning rate
• Temperature τ

A conventional coding agent would just guess. How does the MCP workflow handle it?

1. plan‑parser recognizes this as one block; dependencies: none (leaf node)
2. paper‑search finds three papers—E5 original, DPR, ANCE
3. paper‑extract pulls “mean pooling + weighted average” from E5, and “shared encoder + τ = 1” from DPR
4. spec‑generator produces a spec: pooling=mean, share_weights=True, temperature=0.02, negatives=in_batch+4_hard
5. The agent implements strictly according to the spec, no extras
6. verification‑logger runs a mini‑batch forward test, records loss

Throughout, the engineer only needs to review the spec. Reviewing a spec is ten times lighter than reviewing code.

Limitations of the approach

After all that praise, some caveats.

First, plan quality is everything. If your plan is vague—e.g., “build a killer multimodal model”—the workflow degenerates into the agent guessing. The method assumes an experienced ML engineer, not a vibe coder.

Second, paper‑retrieval noise. Hundreds of new arXiv papers appear daily; precisely finding the few relevant ones per block is nontrivial. The author uses embedding‑based retrieval, but deep learning terms drift—same word, different meaning.

Third, the dependency graph is static. In practice you often realize mid‑implementation “we need A before B.” The current workflow has limited backtracking support.

Fourth, shallow validation. “Runs a mini‑batch” only proves the code doesn’t crash, not that the model’s correct. True verification needs small‑scale training and metric alignment—expensive steps not yet covered.

Takeaways for developers

If you’re building AI‑agent products or systems, this case suggests three lessons:

1. Don’t let the agent make architectural decisions. Humans decide, agents execute. Still iron law in 2026.
2. Break workflows into composable MCP tools. Monolithic agents are obsolete; orchestration is the future.
3. Give agents sources of evidence, not creative freedom. Let them look up justification in papers, docs, or codebases instead of improvising.

As a side note—if you want flexible model switching in such workflows (e.g., plan decomposition with Claude Opus 4.5, paper extraction with Gemini 3 Pro, code implementation with GPT‑5.1 Codex)—OpenAI Hub provides a single key to access models from multiple vendors, OpenAI‑format compatible and connectable from China, sparing you the pain of multiple SDKs. For orchestrated agents matching models to stages, it’s quite handy.

In conclusion

Since Anthropic proposed MCP in 2024, its true value is finally showing—it doesn’t just add tools to AI; it gives engineering workflows a backbone.

This engineer’s open‑source practice represents, in a way, the next stage of AI‑assisted coding: from “letting AI write code” to “letting AI write code by the rules.” The former relies on model capability; the latter on engineering design—and only the latter can truly run in production.

Anyone interested in this direction should check the original post; the author is actively updating it.

References

• An MCP workflow for implementing deep-learning models from an engineering plan – Reddit — Original discussion thread detailing the workflow’s design and usage experience