Qwen Open-Sources QwenGyre for Ultra-Long-Horizon Agent Training

Alibaba’s Qwen Releases QwenGyre, Bringing “Pay by Completion” to Online Reinforcement Learning

On October 12, Alibaba’s Qwen team released QwenGyre, an online reinforcement learning framework designed for ultra-long-horizon agent tasks.

It is not intended to solve the throughput problems found in ordinary model training. Instead, it addresses a problem that emerges as agent tasks become longer: the training system starts being held back by “the slowest trajectory.” A single task may run for several hours, involve hundreds of rounds of interaction between the model and its environment, and consume anywhere from hundreds of thousands to millions of tokens. Traditional online reinforcement learning frameworks either leave large numbers of GPUs waiting for the slowest tasks or completely separate execution from training, resulting in higher latency and wasted resources.

QwenGyre’s core idea is straightforward: start training as soon as a batch of tasks is complete, while unfinished tasks continue running. At the same time, it does not discard content that has already been computed because of mid-task scheduling. In other words, QwenGyre no longer treats a rollout as an orderly formation that must start and finish at the same time. Instead, it treats rollouts as a group of long-running projects that can be settled dynamically.

According to the officially published NL2RepoBench results, on tasks involving approximately 700,000 tokens per rollout, 48 steps of online reinforcement learning increased the model’s success rate from 52.5% to 58.5%. Under the same GPU budget, QwenGyre achieved 1.78x end-to-end acceleration compared with an Async implementation, while its training scores remained broadly comparable to those of Async and Colocate.

This means its value is not simply that it “runs faster.” Rather, it reduces the waiting time wasted in long-horizon agent training without significantly sacrificing training performance.

The Challenge of Long-Horizon Agents Is Not Just Longer Contexts

For chat models, a million tokens may simply represent a very large context window. For agents, however, a million tokens often represents a sequence of real actions: reading code, searching a repository, modifying files, running tests, analyzing errors, modifying the code again, and repeating the process dozens or even hundreds of times.

These tasks have three characteristics.

First, task lengths vary significantly. Some bugs can be fixed within a few minutes, while other tasks require dozens of rounds of exploration in a repository before the root cause is found. Within the same batch of rollouts, the fastest tasks may already be complete while the slowest tasks are still calling tools.

Second, interactions are strongly dependent on one another. Later steps usually depend on the outputs of earlier steps, so the task cannot simply be divided into unrelated short samples. The result of a test command may determine whether the next step is to modify a dependency, revert a commit, or continue tracing the call chain.

Third, training requires complete and accurate trajectories. What the model actually learns is not just the final answer, but also how it plans, when it calls tools, how it handles failures, and whether it can adjust its strategy based on environmental feedback. If only the final result is retained, many intermediate decisions disappear.

Traditional synchronous training waits for all tasks in a batch to finish before entering the next training round. This approach is relatively simple to implement, but even a small number of unusually long tasks can leave other GPUs idle. The longer the tasks and the greater the variation between them, the more severe the idle time becomes.

Async approaches allow execution and training to proceed in parallel, which can usually alleviate some waiting. However, they also face issues such as data freshness, mismatched training and sampling rhythms, and resource contention between different stages. The Colocate approach places execution and training within a relatively tightly coordinated resource layout. This simplifies scheduling, but may not fully utilize resources on ultra-long tasks.

QwenGyre’s approach is to redesign both resource scheduling and trajectory processing, rather than accelerating just one stage.

GPUs No Longer Wait for the Slowest Task: Elastic Scheduling as the First Layer of Change

QwenGyre divides GPUs into multiple compute units. When some rollouts in a batch finish first, those idle units switch to training, while tasks that are still running retain their existing state and continue interacting with the environment.

The key point is not simply “splitting up the GPUs.” It is that task migration does not require starting over. The framework preserves the task’s progress, runtime state, and previously computed caches, then transfers that state as a whole to a new compute unit. This is particularly important for agents with long contexts and extensive interaction histories: if every scheduling operation requires rebuilding the context and recomputing the KV cache, elastic scheduling may merely shift the waste from waiting to recomputation.

This can be understood as a software project involving multiple people working in parallel. Once someone submits code, it can immediately enter code review and regression testing, while those who have not finished continue developing instead of stopping to wait for the entire team. Tasks that finish later can then continue joining the training queue.

This mechanism provides three direct benefits:

• Less GPU idling: Tasks that finish early do not need to wait for the entire rollout batch to conclude and can immediately contribute training computation.
• Higher resource utilization: Execution and training can dynamically occupy available compute units instead of being assigned fixed blocks of resources that may remain idle for long periods.
• Less impact from long-tail tasks: A small number of unusually complex or repeatedly failing tasks will not block the entire training batch.

Naturally, this design also increases system engineering complexity. The scheduler must continuously determine which compute units are suitable for execution and which are suitable for training. It must also coordinate model parameter updates, cache migration, environment-state preservation, and task recovery after failures. For trajectories at the million-token scale, state migration itself is not free.

Therefore, QwenGyre’s significance does not lie in proposing a simple “dynamically allocate GPUs” technique. It lies in treating the execution state of long-horizon agents as a first-class citizen of the training system.

Trajectory Trees and Partial Scoring: Looking Beyond the Final Answer

Resource scheduling addresses how compute is used. Trajectory processing addresses how the resulting interactions are used.

QwenGyre provides a Token-in Token-out recording method that preserves the context actually seen when each segment of output was generated. For agent tasks, this is more detailed than recording only the input, output, and final reward. The context seen by the same model differs across rounds, and tool results, historical actions, and intermediate states all influence the next decision.

The framework also organizes trajectories according to shared prefixes. If multiple execution branches have experienced the same context up to a certain point, they can share that prefix. The system records each distinct path only after an actual branch occurs. During training, it does not expand all branches indiscriminately; instead, it selects the required paths to generate training samples.

This is equivalent to building a “decision tree” for the agent’s working process, rather than crudely compressing every trajectory into a long sequence from start to finish. For tasks such as code repair, terminal operation, and web search, this structure more closely reflects the environment the model actually faces: a single erroneous action may produce multiple subsequent branches, while the final successful path is only one of them.

Another noteworthy mechanism is Partial Scoring, which allows partially completed tasks to receive feedback.

In long-horizon tasks, a timeout does not necessarily mean complete failure. An agent may have correctly located the problem and modified the key files but not had enough time to finish running all the tests. It may also have completed most subtasks but failed at the final stage. If the training system scores tasks only as “ultimately successful” or “failed,” all of this intermediate progress is discarded.

Partial Scoring attempts to turn these partial results into training signals, allowing the model to learn which steps were effective and which errors led to subsequent failure. It cannot replace evaluation of the final result, but it can reduce the sparse-reward problem in long tasks and prevent training from being reduced to only two extremes: success and failure.

What Does 1.78x Acceleration Mean? Do Not Equate It with a Leap in Model Capability

In the official experiments, QwenGyre increased the post-training success rate on NL2RepoBench from 52.5% to 58.5% after 48 training steps. This shows that online reinforcement learning can indeed improve model capability on long-horizon software engineering tasks.

The efficiency metrics are even more noteworthy. Under the same GPU budget, QwenGyre reduced end-to-end training time to approximately 1/1.78 that of the Async implementation. Supplementary data shows that training time fell from 134.55 hours to 75.42 hours, a reduction of approximately 44%. For agent training, metrics like these are more informative than tokens per second alone, because what development teams ultimately care about is whether they can obtain a usable model version faster with the same amount of GPU time.

In cross-task validation using Qwen3.6-122B, QwenGyre achieved end-to-end speedups on all three task categories: NL2RepoBench, DeepSWE, and TerminalBench, with a peak improvement of approximately 1.85x. At least based on the public results, it is not a specialized optimization for just one benchmark.

However, 1.78x should not be interpreted as a 1.78x increase in model inference speed, nor does it mean that the model’s capabilities improved by 1.78x. It primarily reflects the time efficiency of the entire training process, including rollout execution, trajectory processing, GPU scheduling, and parameter updates. The actual benefits will still depend on the distribution of task lengths, environmental response speed, model size, reward function, and GPU topology.

In addition, the public results currently come primarily from software engineering and terminal-interaction tasks. For scenarios such as web browsing, GUI operation, data analysis, and multi-agent collaboration, trajectory branching is more complex and environmental state is harder to preserve. Whether Partial Scoring is equally effective in these settings requires further experimental validation.

QwenGyre’s Real Value: Turning “Long Tasks” into Training Assets

In the past, ultra-long-horizon agent tasks were often treated as a burden on training infrastructure: contexts were too long, runtimes were too extended, failure rates were too high, and rewards were too sparse. Many systems reduced costs by truncating trajectories, splitting tasks into subtasks, or retaining only the final result. But this also weakened the long-term planning capabilities that models genuinely need to learn.

QwenGyre takes the opposite view: long tasks are not inherently noise; they are training data that more closely resembles real work. The key is to make the training system capable of handling this length and of using information that has already been generated before the task is fully complete.

This aligns with the recent direction of development in agent infrastructure. The Qwen platform has already been advancing agent runtime environments, tool calling, long-term memory, and enterprise hosting services. QwenGyre operates at a lower level, addressing how models can continue improving through large volumes of real interaction. Combined, the two could theoretically form a closed loop covering task execution, trajectory collection, automated evaluation, and reinforcement learning updates.

For developers, this direction is more worth watching than yet another short-context benchmark. The future competition between agents will not be determined only by whether a model can answer a question. It will also depend on whether it can run continuously for several hours and still complete a task when encountering errors, tool failures, and environmental changes. Whether a training framework can effectively absorb these long-horizon interactions may become a second dividing line beyond model capability itself.

Assessment: QwenGyre Solves “Training Efficiency,” but Not Yet “Task Reliability”

QwenGyre’s technical approach is pragmatic. It does not focus on a new reward function or a more complex model architecture. Instead, it targets the system bottlenecks most easily overlooked in online reinforcement learning: waiting, migration, caching, trajectory redundancy, and sparse feedback.

Based on the public data, a 1.78x end-to-end speedup is already enough to affect real training costs. This is especially true when training long-horizon agents with models in the 122B class, where saving dozens of hours is often more valuable than optimizing an individual operator. For teams that need to repeatedly experiment with reward design and task environments, shorter training cycles also mean faster iteration.

But QwenGyre is still not the endpoint for long-horizon agents. Once training efficiency improves, the genuinely difficult questions will become more prominent: Are the rewards reliable? Has the model learned exploitative shortcuts? How should partial completion be compared fairly? Can trajectories transfer across different tasks? And how should safety boundaries be controlled during long-running execution?

In other words, QwenGyre is more like clearing a congested training pipeline than directly guaranteeing that the agents produced by that pipeline will be reliable. It makes it economically feasible to train for longer and use more real environmental interactions. Whether those interactions ultimately become stable planning capabilities will depend on whether the evaluation and reward systems can keep pace.

This is also why it deserves attention. As agents move from a few rounds of dialogue toward workflows lasting several hours, the upper limit of model capability will increasingly depend on whether the training system can withstand the complexity of the real world. QwenGyre has at least taken an important step: it no longer treats the most valuable interactions as a system burden simply because the tasks are too long.

References

• ITHome: Alibaba’s Qwen Releases QwenGyre, an Online Reinforcement Learning Framework for Ultra-Long-Horizon Agent Tasks: Introduces QwenGyre’s release, core mechanisms, and official experimental data.
• Qwen AI Platform Model Release Records: Provides background on public updates to Qwen models and the platform.