Karpathy Joins Anthropic’s Pre‑Training Team

On May 19, Andrej Karpathy announced on X that he has joined Anthropic as a member of the pre‑training team. The news came suddenly—just last week he updated his blog with “2025 LLM Year in Review,” discussing the technical evolution of pre‑training, supervised fine‑tuning, and RLHF. No one expected him to jump back into the pre‑training arena so soon.

Why Anthropic

Karpathy’s resume is almost legendary in the AI world. After earning his PhD from Stanford in 2015, he joined OpenAI and worked on early GPT models. In 2017 he moved to Tesla as Director of AI, leading the Autopilot vision team in deploying end‑to‑end neural networks to production cars. After leaving Tesla in 2022, he briefly returned to OpenAI but stayed only a few months before departing again. Since then, he’s been independent—doing education projects, blogging, and podcasting.

His choice of Anthropic over returning to OpenAI makes sense. OpenAI’s current focus is on productization and commercialization. Since GPT‑4, most work has shifted toward alignment, safety, and multimodality—pre‑training itself is no longer the main battlefield. Anthropic, by contrast, is still fighting the hard battle at the foundation‑model level. The scale of Claude 3.5 Opus’s pre‑training, Constitutional AI’s alignment methodology, and the long‑context‑window technical roadmap all demand deep pre‑training optimization.

For Karpathy, that’s a pure technical challenge. His work at Tesla was oriented toward engineering deployment; at OpenAI it was early exploration; at Anthropic he’ll work on pre‑training—joining right at the stage where the approach is validated but still far from its ceiling. At this moment, every decision made by the pre‑training team directly affects the performance ceiling of the next‑generation model.

Anthropic’s Co‑founder Advantage

Karpathy’s addition makes Anthropic a special case: among all top‑tier AI labs, it is the only one whose co‑founding team remains intact. OpenAI’s Ilya Sutskever and Greg Brockman have left; DeepMind’s co‑founders have gone their separate ways; xAI and Mistral are newly founded and still in team‑building phases.

Anthropic already had an exceptionally strong founding lineup: Dario and Daniela Amodei (former VP of Research and former VP at OpenAI), Tom Brown (lead author of the GPT‑3 paper), Chris Olah (a pioneer of interpretability research), and Jared Kaplan (core author of the Scaling Laws paper). With Karpathy now onboard, the company has top experts covering pre‑training, alignment, and interpretability.

More importantly, Anthropic’s technical direction aligns closely with Karpathy’s research interests. His end‑to‑end vision work at Tesla essentially involved training a general‑purpose perception model on massive data, then adapting it to driving scenarios through fine‑tuning and alignment. Applied to LLMs, that’s the same philosophy: build a strong pre‑trained base model, then align it via methods such as Constitutional AI. When Karpathy wrote in his blog that “pre‑training today is like GPT‑2/3 in 2020,” he meant precisely this stage’s importance—the base model’s quality sets the ceiling for everything that follows.

How Much Room Is Left in Pre‑Training

Some might ask: hasn’t pre‑training already maxed out? GPT‑4, Claude 3.5, and Gemini 1.5 all have hundreds of billions of parameters—how much more can be gained by scaling data and compute?

In fact, pre‑training is far from reaching its limit. Scaling Laws show a power‑law relationship among model performance, compute, data, and parameter count—and that relationship still holds at current scales. The issue is that brute‑force scaling is no longer enough; finer work is needed on data quality, training efficiency, and architecture optimization.

Take data, for example: most high‑quality text on the internet has already been used. The next frontier is synthetic data (training data generated by models), multimodal data (images, video, audio), or code and reasoning data. The mix ratios, cleaning methods, and tokenization strategies of each data type all influence final results.

On the efficiency side, today’s large‑model training can take months, and a single failure in any step can ruin the run. How to checkpoint, handle data skew, and maintain numerical stability in distributed training—these engineering issues directly decide whether a model can be trained successfully. Karpathy’s large‑scale training experience from Tesla’s Autopilot program will be highly valuable here.

As for architecture, Transformer has reigned for years, but it still struggles with long context, reasoning, and multimodal integration. Anthropic’s Claude series keeps pushing context length—from 100K to 200K to now 1M tokens—which requires enormous optimization in attention mechanisms, positional encoding, and KV‑cache management. Many of these optimizations must be designed into pre‑training itself; they can’t be patched later via fine‑tuning.

Industry Impact

Karpathy’s arrival will most directly accelerate development of Claude’s next generation. Anthropic has been releasing major versions every six months (Claude 3 in November, Claude 3.5 in May). If the pre‑training team can level up efficiency and quality, that cadence might shorten even further.

More profoundly, his move will intensify the industry’s competition around pre‑training technology. OpenAI, Google, and Meta will recognize that foundation‑model capacity still has headroom—pre‑training remains the core battleground. Expect to see more papers on pre‑training methods, Scaling Laws analyses, and data‑engineering discussions.

For developers, this implies that in the coming year or two, closed‑source models will keep improving significantly. Claude 4—or whatever its name may be—is likely to bring breakthroughs in reasoning, long‑context understanding, and multimodal integration. Such advances will raise the ceiling for AI applications—what’s impossible today may be feasible next year.

Diverging Technical Routes

Interestingly, while Karpathy chose Anthropic, other top researchers are heading in different directions. Ilya Sutskever left OpenAI to found Safe Superintelligence Inc., focused on alignment and safety; Noam Shazeer returned to Google to continue Gemini; Yann LeCun remains at Meta advancing open‑source models.

This divergence reflects different stages and priorities in AI research. Some believe the key is maximizing base‑model capability; others prioritize alignment and safety; still others see open source as the right path. Karpathy’s choice of pre‑training suggests he believes the main bottleneck today is still model capacity itself—not alignment or openness.

That judgment may be correct. Even though GPT‑4 and Claude 3.5 are powerful, they still fall short on complex reasoning, long‑term planning, and multi‑step tasks. Improvements in these areas largely depend on what is learned during pre‑training. If the base model lacks such abilities, no amount of later fine‑tuning or alignment can fully compensate.

In Closing

For Karpathy, joining Anthropic marks a return to his most familiar battlefield; for Anthropic, it completes the final piece of its pre‑training puzzle; for the industry, it signals that the pre‑training race has entered a new stage.

In the coming months, we may see new papers, techniques, and breakthroughs from Anthropic’s pre‑training work—and perhaps early testing of Claude 4, showing striking new capabilities.

For developers, this is a great moment: foundation models are still evolving rapidly, application‑layer possibilities are wide open, and API costs continue to drop. Aggregator platforms like OpenAI Hub already support the entire Claude lineup, allowing direct access with a single key, no network issues.

The war over pre‑training isn’t over—it may, in fact, have only just begun.