GLM Makes Itself Run 3× Faster

Zhipu Brings “AI Improving AI” Into Production

On September 17, Zhipu’s GLM team unveiled its first engineering implementation of Recursive Self-Improvement (RSI): an Infra Agent powered by GLM-5.3 that participated in designing, debugging, and optimizing the inference infrastructure for GLM-5.3-Flash. In less than two weeks, it increased end-to-end throughput to three times the initial baseline.

This system is not merely a lab demo. According to data disclosed by Zhipu, it is already running on a cluster of more than 100,000 domestically produced accelerator chips and handling real-world traffic. GLM-5.3-Flash was previously launched anonymously as Ox-Alpha on OpenCode and OpenRouter, processing more than 62 trillion tokens in six days.

Zhipu says the optimized hardware utilization and per-token cost are now comparable to those of mainstream NVIDIA GPUs. This also marks the first time a Chinese large-model vendor has publicly applied so-called “AI self-improvement” to a production-grade inference system.

What truly deserves attention here is not that “the model wrote a few pieces of code for itself,” but that AI is beginning to enter the most difficult layer of large-model infrastructure—the layer most dependent on experienced engineers: operators, parallelization strategies, GPU memory management, communication topologies, service scheduling, and performance diagnostics.

That said, expectations around “recursive self-improvement” should be tempered. This is still far removed from RSI in the science-fiction sense, in which an AI can autonomously modify its weights, train its successor, and continually raise the upper bound of its intelligence. More precisely, this is a model-driven infrastructure optimization loop: the model helps improve the runtime system that serves the same model family, and the faster system can then support the model in carrying out further engineering tasks at lower cost.

It is not an “intelligence explosion,” but it is already highly practical.

Not Simply Writing Code, but Participating in the Inference Engineering Loop

Traditional coding assistants can also generate CUDA operators, modify configuration files, or analyze logs. But there is a vast engineering gulf between “being able to write code” and “being able to optimize a production cluster with 100,000 accelerators.”

Inference system optimization is rarely a single-point problem. A seemingly ordinary drop in throughput may originate from multiple parts of the system:

• Operators may not be adapted to the chip architecture, leaving compute units underutilized;
• Excessive KV cache usage may reduce concurrency for long-context requests;
• Poor intra-node parallelization may cause communication time to exceed computation time;
• Running prefill and decode workloads together may cause short requests to be slowed down by long prompts;
• A batching strategy focused solely on throughput may cause time to first token to spiral out of control;
• Tail latency in one service component may ultimately amplify into instability across the entire pipeline.

Optimizing one of these areas in isolation may even make another metric worse. For example, increasing batch size generally improves device utilization, but may also increase queuing time. More aggressive quantization can reduce memory usage, but may increase dequantization overhead or impair accuracy on certain tasks.

Information disclosed by Zhipu indicates that the Infra Agent did not merely implement an isolated operator. It helped engineers develop and optimize operators, diagnose performance bottlenecks, and continuously tune the deployment and serving stack. In other words, it took on part of the combined work of a performance engineer, distributed systems engineer, and SRE.

The word “helped” is important here. The currently available public materials are insufficient to prove that the Infra Agent operated entirely without human involvement and autonomously took over a 100,000-accelerator cluster. It would be inaccurate to describe this as “AI independently building an inference system from scratch.” A more reasonable interpretation is that human engineers defined the objectives, constraints, and acceptance criteria, while the Agent executed code changes, testing, log analysis, and multiple rounds of optimization—drastically shortening the trial-and-error cycle.

Even so, increasing end-to-end throughput to three times the baseline in less than two weeks remains a substantial engineering achievement. The most time-consuming part of infrastructure optimization is often not writing the first version of the code, but repeatedly running the cycle of “identify bottleneck—form hypothesis—modify implementation—validate under load—roll back or continue.” This is precisely the kind of work Agents are best suited to take over: work with clear feedback, cumbersome processes, and extensive tool usage.

100,000 Domestic Chips: Scale Is Not the Only Challenge

This time, Zhipu did not focus on peak single-chip compute performance. Instead, it emphasized production deployment on a large-scale cluster of domestically produced chips.

According to the company’s disclosures, these chips are connected through a proprietary high-bandwidth interconnect network, but each individual chip has relatively limited compute power and memory capacity. For an inference system, this means the mature NVIDIA software stack cannot simply be copied wholesale. The relationship among model partitioning, communication, and memory must be reworked.

This is especially relevant because GLM-5.3-Flash supports contexts of up to 1 million tokens. Long context may look like a model specification, but at the deployment level, it is first and foremost a memory problem: the longer the prompt, the greater the prefill computation and the more memory consumed by attention states and caches. When a single accelerator lacks sufficient memory, the system must partition the model and cache across more devices. But as the number of devices grows, communication becomes a new bottleneck.

Zhipu’s technology stack includes:

| Technology | Problem Addressed | |---|---| | Dedicated inference engine built on SGLang | Adapts an existing inference framework to domestic chips and the model architecture | | Intra-node tensor parallelism for linear attention and the LM Head | Partitions large compute modules across multiple accelerators | | ReplaySSM | Optimizes inference for the model’s state-space architecture | | W8A8 quantization | Compresses weights and activations to 8 bits, reducing memory and bandwidth pressure | | Mixed INT8, FP8, and BF16 cache quantization | Selects precision for different stages to balance capacity, speed, and quality | | Layer Split | Partitions the model by layer so larger execution graphs can fit within limited memory | | Disaggregated EPD architecture | Separately schedules multimodal encoding, prompt prefill, and token-by-token decoding |

The item most worthy of developers’ attention is EPD, or Encode–Prefill–Decode disaggregation.

A large-model request does not consume hardware in the same way from beginning to end. Multimodal encoding must process images, video, or audio. The prefill stage reads the prompt all at once and is more oriented toward large-scale parallel computation. The decode stage generates tokens one by one and is more sensitive to memory bandwidth and scheduling latency.

Running all three types of workloads on the same pool of devices is like forcing long-haul trucks, buses, and food-delivery riders to share a single lane: the system may appear unified, but the workloads can easily block one another. EPD separates them into independent worker pools, allowing each to be scaled and scheduled independently. If the number of long prompts suddenly rises, prefill resources can be expanded independently without scaling the decode cluster at the same time.

This type of architecture is not new. The real challenge is making stability, throughput, and latency all reach production-ready levels on domestic chips, with ultra-long contexts, and at a scale of 100,000 accelerators. The value of the Infra Agent lies primarily in jointly exploring a vast space of hardware-related parameters and software strategies.

“3× Throughput” Is Impressive, but Several Key Data Points Are Still Missing

The increase in throughput to three times the baseline is the most eye-catching figure in this announcement, but it should not be interpreted as “the same hardware magically gaining twice as much compute.”

First, the public information does not sufficiently define the baseline. It could have been an initial system without deep optimization, or one that already incorporated basic optimization. The significance of a threefold improvement depends heavily on the baseline used.

Second, end-to-end throughput must be understood in the context of the request mix. Short prompts, long prompts, image inputs, and 1-million-token contexts place very different demands on the hardware. Average output length, concurrency, time to first token, and per-token latency also affect the final result.

Moreover, the statement that “cost has reached a level comparable to mainstream NVIDIA GPUs” still lacks a consistent frame of reference. Is the comparison based on per-card acquisition costs, cluster depreciation, electricity costs, or the total serving cost per million output tokens? Which generation of GPUs is being used, at what precision, and under what service-level requirements? Without this information, the claim is better treated as a directional statement than as a rigorous benchmark.

The figure of 62 trillion tokens also requires further explanation. It is extraordinarily large and may include input tokens, output tokens, cache hits, and internal Agent calls under different accounting methodologies. At the very least, it indicates that the system experienced intensive real-world traffic rather than merely undergoing a few rounds of internal load testing. However, until the accounting methodology is disclosed, it should not be directly compared with the publicly reported token volumes of other API platforms.

For developers and infrastructure teams, the most important question is whether the following metrics will be disclosed:

1. P50, P95, and P99 time to first token before and after optimization;
2. Per-accelerator token throughput at different context lengths;
3. Utilization rates for the prefill and decode stages;
4. Cluster failure, node degradation, and task rescheduling data;
5. How many optimizations proposed by the Agent could be merged directly, and how many required human correction;
6. How much the optimization cycle and staffing requirements were reduced compared with a team of human experts.

If these results can be reproduced, the industry value of this implementation will exceed that of the “China’s first RSI” label.

Why Infra Agents May Create Value Sooner Than General-Purpose Coding Agents

Over the past two years, competition among Coding Agents has focused on fixing GitHub issues, generating applications, and operating terminals. In real-world business environments, however, infrastructure Agents may be more likely to deliver quantifiable returns.

The reason is simple: inference cost is one of the largest variable costs for large-model vendors. The value of a frontend Agent producing a more attractive website may not be immediately measurable. A 20% increase in inference throughput, however, can be translated directly into server counts, gross margins, and API pricing.

Inference infrastructure is also naturally well suited to Agent-based work:

• The target metrics are clear, allowing direct comparisons of throughput and latency;
• Test environments are reproducible, and automated load tests can be run after each change;
• Logs, profilers, and monitoring data provide dense feedback;
• Code changes can be validated through canary deployments;
• The system can automatically roll back when a performance regression occurs.

This is far easier than asking an Agent to determine whether a product requirement is reasonable. The latter depends on ambiguous human preferences, while the former has a relatively objective set of reward functions.

The risks, however, are equally concentrated. An Agent pursuing throughput may achieve higher scores by sacrificing tail latency, stability, or even output quality. If the evaluation metrics are incomplete, it will quickly learn to “optimize for the metrics,” like a student focused only on exam scores, rather than optimizing for production objectives.

A truly mature Infra Agent therefore needs more than code access. It also requires comprehensive safety boundaries: change approvals, resource quotas, sandbox testing, fault injection, automatic rollback, and continuous checks for drift in model quality. When an Agent is allowed to operate a 100,000-accelerator cluster, permission design is no less important than the model’s capabilities.

This Is a Starting Point for RSI, but Not “A Model Evolving Itself”

In the strict sense, recursive self-improvement generally means that an AI can improve its own core capabilities and then use the improved version to carry out the next round of upgrades, forming a continuous recursive process. Zhipu’s publicly disclosed implementation primarily concerns the model-serving system, rather than model weights, training algorithms, or data-generation pipelines.

Moreover, GLM-5.3 is responsible for the optimization, while GLM-5.3-Flash runs on the new system. The two belong to the same model family, but they are not exactly the same model instance. It is therefore accurate to say that “the model participated in building its own runtime environment,” but clearly overstated to claim that “the model can already evolve autonomously.”

Engineering recursion, however, will often emerge before algorithmic recursion. One possible loop is:

• A stronger model generates better inference optimization plans;
• A better system lowers the cost of model calls;
• Lower costs allow the Agent to run more tests and searches;
• More experiments further accelerate system optimization;
• The compute resources saved are used to train or validate the next generation of models.

There is nothing mysterious about this loop, but it could have enormous commercial impact. It directly determines whether a model vendor can offer lower prices, longer contexts, and higher concurrency within the same hardware budget.

From this perspective, the significance of Zhipu’s disclosure is not that it proves “RSI has been achieved,” but that it shows AI beginning to work on its own means of production—not merely generating user-facing content, but modifying the underlying machines that deliver AI services.

Assessment: Forget the Intelligence Explosion for Now—This Already Makes Infrastructure Sense

Zhipu has wrapped this implementation in an RSI narrative with strong publicity value, but the underlying work is substantive. Compared with yet another new model scoring a few points higher on a benchmark, tripling inference throughput and serving large-scale real-world traffic may have a more direct practical impact on the industry.

It also sends a signal: competition among domestic chips depends not only on chip specifications, but also on whether models, compilers, inference engines, and cluster scheduling can be optimized together. A disadvantage in single-chip performance can be offset at the system level through quantization, parallelism, caching, and scheduling—and Agents are shortening the time required for this type of adaptation.

The real dividing line will be whether the Infra Agent can evolve from a successful internal Zhipu project into a reproducible methodology: when given a different chip, model, or traffic pattern, can it still consistently identify effective optimizations without sacrificing reliability?

If the answer is yes, large-model companies may stop upgrading their inference stacks on fixed quarterly schedules. Instead, they may allow Agents to continuously monitor production data, propose patches, and complete canary validation. Inference systems will evolve from manually maintained software into dynamic systems capable of continuous adjustment.

This does not mean the model has suddenly become self-aware. It has simply begun to understand its own profiler data, logs, and resource bills—and for a company consuming vast amounts of compute every day, that may create value far sooner than “self-awareness.”

References

• ITHome: Zhipu GLM Unveils China’s First Recursive Self-Improvement Implementation for Large Models—Reports key information including the Infra Agent, the 100,000-accelerator domestic cluster, the threefold throughput increase, and Ox-Alpha’s real-world traffic.