Anthropic Spends $10 Million to Have Humans Teach AI to Write Code

Anthropic is training Claude Code in a somewhat “counter‑trend” way — by hiring 1,000 real human engineers to do data annotation, paying $280 per task.

The project, codenamed "Marlin," is executed by data annotation company Snorkel AI. According to Business Insider, participating outsourced engineers perform A/B testing on code generated by two different models, selecting the one that better fits real‑world development scenarios and determining whether the model truly understands the technical requirements in the prompt. A task takes about an hour on average, but given the multiple rounds of communication with Snorkel’s approval layer, the actual hourly rate may be less than $280.

Not a Shortage of Data, but a Shortage of the Right Data

Large model companies face a common dilemma: public code repositories have already been scraped extensively, and the marginal benefit from enlarging the training dataset is diminishing. Anthropic’s chosen path this time is refinement — rather than letting the model see more code, it’s teaching it to understand what makes code “good.”

Outsourced participants in the project revealed Marlin’s core goal: train Claude Code to generate “more simplified, easier‑to‑maintain” code. This wording is subtle. In actual development, runnable code and maintainable code are two different things. The former might solve a problem with five layers of nested loops; the latter would break it into three clear functions. Models learn more of the former from GitHub, because the majority of code committed there is “good enough to run.”

Snorkel AI’s background is also worth noting. It’s not a traditional sweatshop‑style data annotation outfit, but a Stanford‑origin team focused on “weak supervision” — leveraging small amounts of high‑quality annotations to handle large‑scale data. Anthropic handing this project to them shows the priority isn’t annotation volume, but annotation quality and the authenticity of engineers’ backgrounds.

$280 an Hour — Worth It?

In the outsourcing market, this rate is medium‑high. Silicon Valley senior engineers’ hourly rates are generally $150–$300, but those are full‑time equivalents including benefits and stability. Marlin project engineers get pure task fees — no benefits, no guarantee of work next month.

The bigger question is scale. If 1,000 engineers worked steadily for a month (8 hours per day, 20 working days per month), the total cost would be $280 × 8 × 20 × 1,000 = $44.8 million. Even if the project runs only a few weeks or not all engineers are fully loaded, the figure easily surpasses $10 million. Compared to the rumored $100+ million compute cost for training GPT‑4, Anthropic’s spending isn’t much, but it’s aimed differently — buying human judgment, not GPU clusters.

This approach isn’t new in the industry. OpenAI’s RLHF (reinforcement learning with human feedback) is essentially hiring people to score outputs, but Anthropic’s distinction here lies in scale and expertise: they’re not hiring just anyone to say “good or bad,” but requiring annotators to be engineers capable of production code, evaluating maintainability, architectural soundness, and other aspects needing experience.

What Exactly Is Claude Code Learning?

Outsourced workers don’t know which version of the model they’re evaluating, but the project design reveals Anthropic’s intent.

Traditional code generation model training is more like “fill‑in‑the‑blank”: given context, predict the most likely next line of code. This teaches syntax and common patterns, but not why a particular implementation is better. Marlin’s A/B testing essentially teaches the model multiple‑choice reasoning: given two functionally correct code snippets, which better fits engineering practice?

This aligns the model with developers’ tacit knowledge. For example:

• Naming Conventions: Should a variable be userData or ud? The former is verbose but clear; the latter is concise yet unreadable after three months.
• Error Handling: Wrap the whole function in a try-catch, or catch errors at critical points? The former is easier but creates debugging nightmares; the latter is tedious but controllable.
• Level of Abstraction: Should 5 lines of code be abstracted into a function? Premature abstraction is harmful, but after copy‑pasting three times, refactoring is warranted.

These judgment criteria aren’t clearly explained in textbooks, nor easily quantified on Stack Overflow; only experienced engineers can weigh them case‑by‑case. Anthropic is now systematically instilling this weighing ability into Claude Code.

Developer Fit Is a Real Issue

The biggest complaint about code generation tools these past two years isn’t “can’t write,” but “writes something I didn’t want.” Early GitHub Copilot often auto‑completes large blocks, forcing developers to check line‑by‑line for pitfalls. Earlier versions of Claude Code had similar issues: given a complex request, it might output a seemingly complete solution, but on closer inspection some edge cases aren’t handled, some dependencies are wrong.

This is a “fit” problem — the model’s mental mapping doesn’t match the developer’s. When a developer says “write a user authentication module,” they might envision JWT + Redis cache + permission middleware as the standard setup. The model might write a session‑based implementation, or use OAuth2 and overcomplicate it. Technically correct, but not the optimal solution for this scenario.

If Marlin succeeds, Claude Code will better understand these unwritten rules. It’ll know in which tech stack developers expect certain defaults, when to be conservative (e.g., in financial systems), and when to be aggressive (internal tools).

Who’s the Target?

GitHub Copilot holds the largest market share, but its training relies more on GitHub’s immense code corpus. The advantage is breadth; the drawback is variability — you never know whether a learned pattern comes from Google’s internal tooling or a student’s homework.

New players like Cursor and Codeium take an engineering‑focused route: better IDE integration, more precise context understanding, faster response times. They compete sharply on interaction experience, but their underlying model capabilities still depend on OpenAI or Anthropic.

Anthropic’s current strategy is a sort of “dimensionality‑reduced strike.” While others iterate product form, Anthropic is addressing code quality at the model layer. If Claude Code can consistently output “more maintainable” code, its appeal in the enterprise market will rise significantly — after all, technical debt is the CTO’s number one enemy.

Can This Approach Last?

In the short term, the Marlin project is effective. Human engineers’ judgement can quickly calibrate the model’s output preferences, bringing it closer to “senior developer” code taste within months.

But in the long run, this approach has scale limits. There’s no absolute standard for good or bad code; preferences differ across companies, teams, and project stages. Google’s prized code style might be over‑engineering for a startup. Can the preferences of these 1,000 engineers represent the whole developer population?

The bigger question: if the model already understands code well enough, why does it still need humans to teach it what “good code” is? This hints at limitations in current architectures — able to memorize patterns, but unable to grasp principles.

Anthropic may be betting on an intermediate stage: models aren’t yet smart enough to grasp engineering aesthetics on their own, but smart enough to learn quickly from human feedback. If this window is long enough, spending tens of millions for human judgment is worth it. If the next generation leaps past this stage, Marlin is merely a transitional plan.

What Should Developers Worry About?

Not losing their jobs — at least not yet.

The Marlin project actually highlights that AI’s greatest gap now isn’t the ability to write code, but the ability to understand why it’s written that way. As long as this gap exists, human engineers have value.

But it does compress the space for a certain type of work: mechanical, pattern‑based coding tasks. If your day‑to‑day job is to turn product requirements into standard CRUD operations, then when Claude Code evolves to the next stage, your competitiveness will weaken.

Conversely, if your value lies in architectural decisions, tech stack selection, performance optimization, and cross‑team coordination — tasks that require context and judgment — AI won’t replace you in the short term. What Marlin engineers are doing now — evaluating code quality, weighing technical options — is exactly the part AI can’t yet learn.

Ironically, Anthropic is proving, while training AI with humans, that some human abilities are still irreplaceable — $280/hour annotation fees are, in a sense, the market pricing for this gap.

The Endpoint of Data Annotation

This project also reflects a deeper contradiction in large model training: the further you go, the bottleneck isn’t compute, but data.

OpenAI, Anthropic, and Google have nearly scraped the entire internet. The next step is either to buy proprietary data (copyright issues), synthesize data (quality issues), or conduct large‑scale manual annotation like Marlin (cost issues). None of these routes are easy.

Snorkel AI’s business model is interesting — it essentially productizes and services the vague concept of “data quality.” Data annotation used to be sweatshop labor; now it’s “engineers‑as‑a‑service.” When the day comes that models no longer need human feedback, this sector will have to find new value.

From this angle, Marlin is both Anthropic’s tactical move to catch up in the code generation race, and an industry‑wide strategic experiment in this “post‑pretraining era” — everyone is exploring where the next scalable quality improvement will come from once public data runs out.

Anthropic’s answer: let those who know code best teach AI what good code is. It’s a simple and blunt logic, but it may work. How long it lasts depends on when the next generation of models can figure it out themselves.

---

References

• ITHome – Anthropic Reportedly Hires 1,000 Human Engineers to Train Claude Code — Project details including pricing and workflow