AI Rewrites COBOL—and Migrates the Bugs Too

AI Can Translate Code, but It Cannot See the Entire System

A study released in late July brought a growing enterprise demand back down to earth: using AI to migrate legacy COBOL programs to Java can indeed significantly accelerate code rewriting, but the generated results still contain bugs—and may even introduce semantic deviations that are harder to detect than compilation errors.

The problem is not that the Java is poorly written. Quite the opposite: AI-generated Java is often well structured, follows naming conventions, compiles successfully, and may even pass every unit test. The danger is that it may correctly translate only the code in front of it without understanding the role that code plays in the broader mainframe system.

A COBOL program that has been running for thirty years is rarely just a standalone program. It is typically connected to COPYBOOKs, JCL jobs, VSAM files, DB2 tables, CICS transactions, overnight batch processing, reporting systems, and manual ledger adjustment procedures. AI sees source code; what actually runs is a dependency network spanning decades.

This is also the most important takeaway from the new research for developers: syntax conversion is no longer the hardest part of migration—behavioral equivalence is.

Successful Compilation Only Proves That the Compiler Has No Objections

In the past, enterprises migrating COBOL generally relied on manual analysis, rule-based converters, and long periods of parallel operation. The most obvious change after generative AI entered the process is speed: logic that once had to be rewritten section by section by engineers can now be converted into Java by a model in minutes.

But between “can be generated” and “can go into production” lies an entire validation engineering process.

Field conversions like the following represent the most basic type of risk:

01  ACCOUNT-BALANCE  PIC S9(7)V99 COMP-3.
01  INTEREST-RATE    PIC 9V9999.
COMPUTE ACCOUNT-BALANCE =
    ACCOUNT-BALANCE * (1 + INTEREST-RATE).

AI can easily rewrite it as:

double accountBalance;
double interestRate;

accountBalance = accountBalance * (1 + interestRate);

The code compiles and runs, but this is generally not a reliable migration of financial semantics. COBOL fixed-point decimals, packed decimals, field lengths, truncation, and rounding rules are not equivalent to Java binary floating-point numbers. Monetary calculations should use BigDecimal, with an explicitly defined scale, rounding mode, overflow strategy, and database field mapping.

Even if the model uses BigDecimal, the problem is not solved. The legacy program may depend on the behavior of automatically truncating a value when it is assigned to a fixed-length field, while a downstream report may treat that truncation as the de facto standard. If AI unilaterally adopts more reasonable rounding, the code quality may appear to improve, but the accounts will no longer reconcile.

This kind of bug will not crash the system at startup. It may create a discrepancy of only a few cents per day and remain undetected until a general-ledger reconciliation six months later. For banking, insurance, tax, and aviation systems, this is far more dangerous than an explicit exception.

The Real Semantics Often Are Not in the COBOL File

When AI migrates legacy systems, it is most likely to miss four categories of context.

1. Data Representation Is Not Ordinary Type Mapping

COBOL contains many data structures with runtime semantics, including:

• COMP-3 packed decimals;
• REDEFINES, which provides different interpretations of the same memory area;
• OCCURS DEPENDING ON variable-length arrays;
• Level-88 condition names;
• Space padding, leading zeros, and special sentinel values in fixed-width files;
• Encoding differences between EBCDIC and ASCII or UTF-8.

If REDEFINES is translated simply into two Java fields, the generated code may appear more “object-oriented,” but it loses the original semantics of shared storage. Similarly, all-spaces, all-zeroes, low values, and high values in legacy systems do not necessarily map directly to Java’s null.

2. Call Relationships Are Hidden Outside the Source Code

In modern Java projects, most call chains can be found in packages, classes, methods, and dependency injection configurations. That is not necessarily true of mainframe systems.

Whether and when a COBOL program runs, and which dataset it receives as input, may be determined by JCL. Modules may be connected through dynamic CALL statements, file drops, or message queues. A particular return code may not indicate an error at all, but instead instruct the next job to take a different branch.

If AI reads only a source directory, it may produce a result that is locally correct but globally wrong. For example:

ProcessPayment
  -> Writes an intermediate file
  -> JCL evaluates the return code
  -> Overnight job reads the file
  -> Updates the general ledger
  -> Reporting job generates a regulatory file

If the model sees only the first step when rewriting ProcessPayment, it may replace the intermediate file with a database write, considering this a reasonable modernization. But the overnight job is still waiting for that file, and the entire chain then fails silently.

3. Historical Bugs May Have Become Business Rules

Legacy systems often contain strange logic that no one dares to touch. It may be an old bug, or it may be a patch added for a specific customer, region, or regulatory rule.

If a migration tool “optimizes” such code along the way, a difficult problem arises: the new implementation may be more correct from an engineering perspective, yet its business behavior differs from the production system.

A COBOL-to-Java migration therefore cannot ask only, “Does this logic make sense?” It must also ask:

How exactly has the production system performed this calculation for the past thirty years? Do other systems already depend on the result?

Before business owners confirm otherwise, AI should not independently correct any branch that appears redundant.

4. Nonfunctional Behavior Is Also Part of Compatibility

Batch windows, lock granularity, transaction boundaries, file-sort stability, failure rerun mechanisms, and throughput are all part of a migration.

A COBOL program may process tens of millions of records in one nightly run and commit at specific checkpoints. After AI converts it into a conventional Spring service, its business results may be entirely correct on a small dataset, yet in production it may fail to complete within the batch window because of per-record commits, heap growth, or database lock contention.

Therefore, “same input, same output” is not enough. Execution order, performance boundaries, and failure recovery behavior must also remain acceptably consistent.

Lower Complexity Does Not Mean Lower Risk

Some earlier studies of AI-driven migration used McCabe cyclomatic complexity, abstract syntax tree node counts, and module coupling to demonstrate the effectiveness of generative migration. AI can indeed split a massive COBOL procedure into multiple Java classes and methods, making the code look shorter and clearer.

These metrics are valuable, but they cannot be treated as evidence of a successful migration.

Cyclomatic complexity measures the branching structure of code; it does not measure whether business semantics are complete. If a model removes exception branches, boundary conditions, and historical compatibility logic, complexity will naturally decrease. The problem is that what it removed may be precisely what allowed the system to operate reliably for thirty years.

In other words:

• Cleaner code does not mean more correct behavior;
• Fewer dependencies do not mean that dependencies were correctly replaced;
• Higher unit test coverage does not mean that real production data is covered;
• Java compiling successfully does not mean that the COBOL migration is complete.

If the migration team treats the “code generation success rate” or “compilation pass rate” as a core KPI, the project may easily be declared complete at precisely the point when optimism is least justified.

GraphRAG Is More Useful Than a Long Context Window, but It Is Not a Silver Bullet

To address missing context, one emerging approach is to build a knowledge graph for the legacy system and then use GraphRAG to provide dependency information to the model.

Conventional vector-based RAG is good at finding semantically similar code. For example, a query for “interest calculation” can retrieve programs containing INTEREST, RATE, and BALANCE. But the critical questions in legacy systems usually require multi-hop reasoning: if the interest-rate logic in module A is modified, which JCL job, DB2 table, reporting program, and ultimately which regulatory file will be affected?

Such questions are difficult to answer through similarity search alone. A knowledge graph can explicitly store relationships:

COBOL program --CALLS--> Subprogram
COBOL program --COPIES--> COPYBOOK
JCL step      --EXECUTES--> COBOL program
Program       --READS--> VSAM dataset
Program       --UPDATES--> DB2 table
Report        --DEPENDS_ON--> DB2 table

Before generating Java, the model can traverse related nodes in the graph and obtain not merely “similar code,” but the complete chain that the modification may affect. This is more reliable than simply stuffing hundreds of thousands of lines of source code into a long context window: no matter how large the context window is, it will not automatically turn implicit relationships into a verifiable dependency graph.

However, GraphRAG cannot solve every problem. The graph itself may be incomplete, and it may omit dynamic calls, runtime-generated filenames, manual operations, and external vendor interfaces. Relationships that static analysis cannot discover must be supplemented with runtime tracing, job logs, database audits, and business interviews.

A sound architecture should:

1. Use static analysis to extract programs, files, tables, and call relationships;
2. Supplement execution chains using JCL, CICS configurations, and scheduling platforms;
3. Validate actual calls using production logs and distributed tracing;
4. Add business terminology, historical change requests, and manual rules to the graph;
5. Only then allow the model to generate or modify code based on that graph.

AI should come after dependency analysis, not before it.

A More Reliable Migration Process: Prove First, Then Replace

If a team genuinely wants to use AI for COBOL migration, translating the entire repository in one shot is the worst possible approach. A more realistic process can be divided into six steps.

Step 1: Build an Asset Inventory

First determine what the system actually contains rather than immediately generating Java. At a minimum, the inventory should cover:

• COBOL programs, COPYBOOKs, and shared subprograms;
• JCL, scheduled tasks, and upstream and downstream jobs;
• VSAM, sequential files, database tables, and message queues;
• CICS transactions, external interfaces, and points of manual operation;
• Production invocation frequency, failure rate, and business criticality.

Dead code that has not run for years and programs that process core accounts every day should not follow the same migration strategy.

Step 2: Establish a Golden Master

Before modifying the code, record the legacy system’s actual behavior. Select sanitized production samples, historical boundary data, and exceptional data, then preserve inputs, outputs, return codes, database changes, file contents, and logs.

These results form the Golden Master. It does not guarantee that the legacy system’s logic is “correct,” but it can demonstrate whether the new system has changed existing behavior.

Step 3: Have AI Generate Tests Before It Generates the Implementation

Rather than translating directly, AI is better suited to first explaining code, enumerating execution paths, and generating test scaffolding. Engineers can then use these outputs to check whether the model understands:

• Monetary and date boundaries;
• Special status codes;
• Empty files and duplicate records;
• Batch interruptions and reruns;
• Database commit and rollback conditions.

If the test plan cannot be clearly explained, the model should not be allowed to continue generating a production implementation.

Step 4: Perform Differential Testing

Run the COBOL and Java implementations in parallel using the same inputs, and compare the results field by field. Monetary systems must compare not only final balances, but also intermediate calculations, rounding results, and accounting entries.

For batch systems, teams must also compare record order, file length, checksums, return codes, and recovery positions after failures.

Step 5: Use Shadow Operation Instead of Switching Traffic Directly

The new Java system can initially read a copy of production traffic without performing external writes, or it can write its output to an isolated environment. After continuously comparing results for weeks or even months, the team can gradually enable read-only queries, low-risk customers, or a small percentage of business traffic.

For the most critical accounting writes, the ability to switch back rapidly to the legacy system must be retained.

Step 6: Perform Data-Level Reconciliation

Monitoring cannot focus only on error rates and latency. Hidden bugs often do not throw exceptions, so business invariants must also be monitored:

• Do debits and credits balance?
• Are end-of-day totals consistent?
• Has the record count changed abnormally?
• Is the discrepancy between Java and COBOL results continuing to grow?
• Have state combinations appeared that were never seen before?

The truly dangerous migration incident is not a service returning HTTP 500 errors. It is a service returning HTTP 200 every day while slowly corrupting the database.

AI Is Well Suited to Being an Accelerator, Not the Final Arbiter

This does not mean that AI should not be used for COBOL modernization. On the contrary, it is highly useful for explaining legacy code, renaming variables, performing structural refactoring, generating tests, filling documentation gaps, and converting low-risk modules. It can also help younger Java engineers quickly understand poorly documented COBOL programs, alleviating the mainframe skills gap.

But teams need a clear understanding of its limitations:

AI is good at translating local code, but not at independently discovering every implicit constraint in an enterprise system.

If complete dependencies, business rules, and test samples are already available, AI can dramatically shorten the migration timeline. If these foundational materials do not exist, the model will merely produce a system that “looks complete” more quickly.

For ordinary internal reports, this may only mean that the numbers fail to match. For payment, clearing, insurance policy, tax, and public service systems, the result may be months of ongoing data corruption.

Therefore, when evaluating AI migration tools, development teams should not look only at how many lines of code were converted in a demo. They should ask three questions:

1. How does the tool discover dependencies outside the source code?
2. How does it prove behavioral equivalence between Java and COBOL?
3. When deviations occur, can they be located and reconciled, and can the system be rolled back quickly?

If those questions cannot be answered, greater conversion speed may mean greater risk.

Conclusion

As of August 2026, AI has reduced COBOL-to-Java syntax conversion from the “main engineering effort” to a “foundational step,” but it has not eliminated the most expensive parts of legacy system modernization: system discovery, business validation, differential verification, and production governance.

The warning from the latest research is not that “AI cannot write Java,” but that it can too easily write Java that looks like the correct answer. Compilers check only syntax and types. Unit tests check only the scenarios the team has anticipated. What truly determines whether a migration succeeds or fails is the historical semantics that were never written into the prompt, the documentation, or even the source code.

AI can be responsible for rewriting the code, but humans must still be responsible for proving that the rewrite has not broken the business.

References

• Original study: AI migrated legacy COBOL programs to Java, bugs included, arXiv 2607.28271, released in July 2026. Due to restrictions on external link domains at the end of this article, no external link is included here. The study discusses hidden defects introduced by automated AI migration.
• AI Killed COBOL—Who Is Next? Lessons from Anthropic vs. IBM: Discusses the impact of AI-driven reductions in legacy system migration costs on the mainframe ecosystem, enterprise software moats, and the migration market.