Research Code, Now Being Refactored by Agents

Recently, OpenAI released a practical report on scientific computing in the age of agents. It does not discuss having large models write a few Python functions for researchers. Instead, it addresses a more specific and more difficult issue: AI coding agents are entering real-world scientific codebases, helping modernize legacy software, and compressing engineering work that once took months into much shorter cycles.

The report covers scientific computing scenarios such as genomics. Coding agents like Codex can read codebases, modify files, run tests, fix errors, and continue iterating based on execution results. Their role is changing accordingly—they are no longer merely autocomplete tools inside an IDE, but more like remote collaborators that can take ownership of well-defined engineering tasks while still requiring rigorous validation.

This development deserves attention not because researchers are finally using AI for programming, but because scientific software happens to be one of the most challenging—and most valuable—arenas for coding agents.

The Problems With Scientific Software Have Never Been Just About “Old Code”

A great deal of scientific computing software is nothing like a standard internet product.

It may have been maintained continuously by a laboratory for more than a decade, mixing Fortran, C, C++, Python, R, Shell, and CUDA. It may depend on specific versions of compilers, mathematical libraries, operating systems, or even cluster scheduling environments. Tests may be incomplete, while documentation is scattered across papers, emails, and researchers’ personal experience.

More importantly, scientific code usually has two responsibilities:

1. Implement a particular scientific method;
2. Ensure that implementation runs reliably on real-world data and computing resources.

Traditional software refactoring is highly sensitive to changes in interface behavior. Scientific software must guard against an additional risk: the code may run successfully while the scientific results have quietly changed.

For example, migrating an old sequence-analysis pipeline to a new dependency stack is not successful merely because it compiles. Changes in floating-point precision, random seeds, parallel execution order, default parameters, or even file-parsing boundaries can alter the final results. For an ordinary web service, this may amount to an inconspicuous output difference; for genomic analysis, climate simulation, or materials computation, it may affect the conclusions of a research paper.

Therefore, “scientific software modernization” is not simply converting Python 2 to Python 3, nor is it asking an agent to replace deprecated APIs in bulk. At minimum, it includes:

• Identifying implicit dependencies and build environments;
• Filling gaps in unit tests, integration tests, and regression samples;
• Updating language versions, compiler toolchains, and third-party libraries;
• Decoupling overly entangled modules;
• Keeping numerical results, statistical properties, and performance boundaries within acceptable limits;
• Leaving future researchers with reproducible build and execution methods.

In the past, this kind of work often went undone because no one wanted to do it. PhD students would rather develop new methods than spend months organizing build scripts left behind by their predecessors, while professional software engineers might not understand the scientific constraints behind the algorithms. The value of AI coding agents emerges precisely in the gap between the two.

What Agents Really Change Is Task Granularity

The basic unit of interaction with a traditional coding assistant is a piece of text: a developer asks a question, and the model returns an explanation or code snippet. The human still has to modify files, run commands, diagnose errors, and provide additional context.

Coding agents elevate the unit of interaction to an engineering task. For example:

Migrate this analysis tool to the new compiler version while preserving the existing command-line interface. First establish baseline tests, then fix failing cases one by one, and finally provide a performance comparison and a list of unresolved risks.

This cannot be completed in a single generation. The agent must repeatedly “read—plan—modify—run—observe—modify again” within the codebase while processing build logs, test results, and version-control diffs.

From a workflow perspective, it more closely resembles the following loop:

Define scientific constraints and acceptance criteria
          ↓
Agent reads the code, paper documentation, and build files
          ↓
Establish a reproducible environment and baseline results
          ↓
Modify code, dependencies, and tests in batches
          ↓
Run regression tests, numerical validation, and performance tests
          ↓
Researchers review differences and their scientific implications
          ↓
Merge, or begin another round with feedback

The most important question here is not whether the agent can write a particular language, but whether it can obtain sufficiently complete feedback. Compilers, testing frameworks, static analyzers, and benchmarks provide structured signals that allow the agent to keep correcting itself. Without such feedback, so-called autonomous development can easily degrade into “continuously generating large numbers of plausible-looking patches.”

This is also why scientific computing is both well suited to agents and prone to amplifying their shortcomings. On one hand, research projects contain many mechanical yet cross-file maintenance tasks that are suitable for automation. On the other, scientific correctness often cannot be summarized by a simple test passed.

Passing Tests Does Not Mean the Scientific Conclusions Remain Unchanged

The central message of OpenAI’s report is not that “agents can replace scientific programmers,” but that research teams need to redesign their validation systems.

A more reliable acceptance stack usually has at least four layers.

Layer 1: Engineering Correctness

This includes whether the code compiles, commands execute, interfaces remain compatible, and error paths behave properly. This layer most closely resembles traditional software engineering and is also the easiest for agents to handle.

Layer 2: Numerical Consistency

Do the old and new versions produce identical—or sufficiently similar—results for fixed inputs? Simple string equality is not enough here. Absolute-error, relative-error, and distribution tolerances must be defined according to the properties of the algorithm.

Layer 3: Scientific Validity

Even if there are reasonable numerical differences, do the outputs still satisfy domain constraints? For example, are conserved quantities preserved, are statistical metrics stable, and are known positive samples still detected? Domain experts must define the standards at this level; models cannot be left to guess them.

Layer 4: Performance and Reproducibility

Correct results are generally still unacceptable if runtime increases tenfold. Research teams must also examine CPU, GPU, memory, and I/O overhead, as well as whether results can be reproduced on another machine or in another container.

A task description suitable for an agent should not say only “upgrade the dependencies.” It should include these constraints:

task: modernize-pipeline
constraints:
  cli_compatibility: required
  numerical_tolerance:
    relative_error: 1e-6
  benchmark:
    runtime_regression: "< 5%"
    peak_memory_regression: "< 10%"
  reproducibility:
    container_build: required
    fixed_seed_cases: required
  review:
    domain_expert_approval: required

There is nothing magical about this configuration itself. The real change is that it turns acceptance criteria previously stored in the minds of senior researchers into an engineering contract that both machines and teams can read.

Why Genomics Has Become a Representative Use Case

It is no surprise that the report specifically mentions genomics.

The field involves massive datasets and long processing pipelines, typically linking multiple stages such as sample quality control, alignment, variant calling, annotation, and statistical analysis. Tool versions, reference data, parameters, and file formats are interdependent, and upgrading any single component can trigger cascading effects.

At the same time, genomics has accumulated a wealth of public algorithms, tools, and data formats. For coding agents, this means access to relatively rich code context. For teams, it means many engineering issues can be validated using public test samples and standardized workflows.

Tasks suitable for agent involvement include:

• Fixing compatibility issues in legacy tools on new compilers or operating systems;
• Adding boundary cases to parsers that lack sufficient tests;
• Consolidating scattered Shell pipelines into traceable workflows;
• Updating deprecated dependencies while retaining outputs from old versions as regression baselines;
• Analyzing performance hotspots and experimenting with parallelization or reductions in unnecessary I/O;
• Generating containers, continuous integration configurations, and deployment documentation.

However, decisions involving algorithm definitions, sample selection, statistical assumptions, and clinical significance still cannot be outsourced merely because an agent can submit code. A model may help identify inconsistencies between an implementation and its documentation, but it cannot inherently determine which one represents the correct scientific interpretation.

The Speedup Is Real, but Do Not Mistake Tokens for Productivity

OpenAI’s recently disclosed internal usage trends also help explain the context behind this scientific computing report. By June 2026, among OpenAI’s active internal users, the median number of output tokens generated with Codex in the research organization had increased by approximately 56 times compared with November 2025—higher than the roughly 27-fold increase in the engineering organization over the same period.

This indicates that coding agents are spreading from engineering teams into research work. But one point requires emphasis: growth in token usage is not growth in productivity, much less growth in scientific discovery.

Agents naturally consume more tokens as they read large volumes of logs and repeatedly modify code. More meaningful metrics should include:

• How much shorter is the cycle from identifying a problem to producing a verifiable patch?
• Have legacy projects improved their test coverage and reproducibility?
• Are researchers spending less time on environment configuration and dependency conflicts?
• Can regressions and errors introduced by agents be caught automatically?
• Is software maintenance becoming less dependent on a single team member who may soon leave the group?

If a team merely replaces a chat window with an agent without adding tests, baseline data, and review processes, it will probably end up with nothing more than technical debt produced at a faster rate.

Research Teams Need Better “Scaffolding,” Not Longer Prompts

For developers, the report’s most useful lesson is not to treat agent adoption as a prompt-engineering exercise.

What often determines effectiveness is whether the repository itself is suitable for machine-driven work: Are build commands standardized? Can dependencies be pinned? Can tests run automatically? Are failure logs clear? Is the task scope sufficiently narrow? Are domain rules documented and encoded as assertions?

A practical sequence for adoption is:

1. Start by giving agents reversible tasks. Begin with documentation, tests, build configurations, and small compatibility fixes rather than immediately rewriting core algorithms.
2. Establish a baseline before allowing changes. Freeze outputs from the old version, performance data, and known anomalies to prevent the agent from “optimizing” without a reference point.
3. Limit the scope of each change. Make each task correspond to one clearly defined problem, and require a description of the diff, testing evidence, and unresolved risks.
4. Turn domain knowledge into assertions. Rules that can be expressed as tests should not exist only in papers or verbal explanations.
5. Keep final human sign-off. Code reviewers assess engineering quality, while domain experts assess scientific meaning. Neither can replace the other.

This approach may appear conservative, but scientific code is not suited to pursuing “zero human involvement.” Agents can take on large amounts of execution work, while humans focus their attention on defining objectives, judging anomalies, and interpreting results. The truly scalable model does not remove people—it places human judgment where it is most costly and most critical.

Assessment: Agents Will Rescue Legacy Code Before They Accelerate New Discoveries

“AI accelerating scientific discovery” is a story that is easy to overstate. By contrast, the path presented in OpenAI’s practical report is more pragmatic: use coding agents first to address the long-unpaid engineering debt in scientific infrastructure.

This may not sound romantic, but it could produce an impact more quickly. Adding tests to a critical tool that has been impossible to upgrade for years and that only its original author dares to modify, migrating it to a modern environment, and making its builds reproducible may be more valuable than having a model generate another hundred experimental scripts.

At present, the most reliable capability of coding agents is not independently proposing scientific theories, but accelerating engineering loops in which the objectives are clear, the feedback is actionable, and the results are verifiable. Scientific computing happens to offer a large number of such tasks, though its acceptance standards are stricter than those of ordinary software.

The dividing line in this transformation, therefore, will not be which laboratory was first to buy Codex, but which one is first to convert scientific knowledge into executable tests, benchmarks, and constraints. Without a validation system, an agent is merely a fast-moving risk amplifier; with one, it can become a force multiplier for modernizing scientific software.

References

The core facts in this article come from OpenAI’s recently released practical report, Scientific Computing in the Age of Agentic AI. Due to restrictions on external-link domains at the end of the article, no link to the original site is included here.

• OpenAI Codex GitHub repository: Learn about the public implementation, operation, and development progress of the Codex coding agent.
• OpenAI Cookbook: Contains publicly available practical materials on model tool use, evaluation, and agent workflows, and can serve as a supplementary reference for building validation processes.