Qwen3.8 Tops the Agent Leaderboard

Qwen Takes the Global No. 1 Spot in Agent Capabilities for the First Time

On August 6, Artificial Analysis released its latest model rankings. Alibaba’s Qwen3.8-Max topped the Agentic Index leaderboard with a score of 55.4, surpassing Claude Opus 5 and GPT-5.6.

This is not the first time a Chinese model has outperformed overseas models in an individual benchmark, but the agent capabilities leaderboard carries a different kind of weight.

Over the past two years, models from China and abroad have taken turns posting high scores on static benchmarks covering mathematics, coding, question answering, and other areas. The Agentic Index, however, does not focus on whether a model can “get the answer right on the first try.” Instead, it evaluates whether the model can keep advancing a task in a complex environment: breaking down objectives, calling tools, reading feedback, revising its approach, and ultimately completing dozens or even hundreds of steps.

In other words, traditional benchmarks are more like closed-book exams, while agent evaluations are closer to asking a model to work independently.

The top positions on this leaderboard had long been dominated by the Claude and GPT families. The previous best result for a Chinese model was 50.1, achieved by Moonshot AI’s Kimi K3. By raising the score to 55.4, Qwen3.8-Max has not only set a new record for Chinese models but also taken the global No. 1 position for the first time.

One point that is easy to confuse must first be clarified: when people say that “Qwen3.8 topped the agentic rankings,” the model actually compared on the leaderboard was the flagship Qwen3.8-Max; the score of 55.4 refers to the Agentic Index. Another figure often cited alongside it, 56, is Qwen’s score on the Artificial Analysis Intelligence Index. These are different metrics and should not be conflated.

What Does a Score of 55.4 Mean?

Agentic capability is not a standalone skill, but the product of multiple capabilities.

A model may be excellent at writing code but forget the original objective after its fifth tool call. Another may appear to reason comprehensively but fail to adjust its strategy when a webpage’s structure changes, an API returns an error, or a file format is abnormal. For real-world agents, these problems are often more consequential than getting one fewer math question right.

A usable agent system must be able to handle at least several things simultaneously:

• Planning: Break ambiguous objectives into ordered, verifiable subtasks;
• Tool use: Correctly choose among search, browsers, code executors, and internal enterprise systems;
• State management: Remember what has already been completed and what comes next, even after running for an extended period;
• Feedback-driven correction: When a tool returns an error, understand the problem and change strategies instead of simply retrying in place;
• Result validation: Check whether the deliverable truly satisfies the objective rather than merely generating a summary that looks complete.

A score of 55.4 therefore should not be interpreted simply as a measure of “how much smarter Qwen is than Claude.” More precisely, it indicates that, within the task set, tool environment, and scoring framework currently used by Artificial Analysis, Qwen3.8-Max has the highest overall success rate in completing complex tasks.

This is also the most significant aspect of its rise to the top. Chinese models have traditionally enjoyed advantages in pricing, Chinese-language capabilities, and deployment flexibility. Competition is now moving into end-to-end task execution: models are no longer merely providing answers, but directly taking responsibility for workflows.

This Improvement Did Not Come From Making the Model More “Talkative”

Qwen3.8-Max is the largest and most capable flagship model in this generation of the Qwen family. According to information released by the Qwen team, the model uses a sparse mixture-of-experts architecture, scaling its total parameter count to 2.4 trillion, with approximately 95 billion parameters activated per inference, while employing a hybrid attention mechanism to improve long-context efficiency.

The basic idea behind this design is not difficult to understand: the 2.4 trillion parameters are like a vast team of experts, but not everyone is called into a meeting for every task. Instead, the model activates the “experts” corresponding to approximately 95 billion parameters based on the problem at hand. This expands model capacity without causing the cost of each inference to grow linearly with the total parameter count.

Of course, MoE does not inherently guarantee better performance. The final result depends on whether routing is accurate, whether different experts truly develop distinct specializations, and whether training maintains balanced loads. What makes Qwen3.8-Max noteworthy is that its large-scale MoE architecture has not merely translated into broader knowledge coverage, but also into stronger tool use, long-horizon planning, and task-correction capabilities.

The model also supports context windows of up to 1 million tokens, along with visual understanding. Long context is important for agents, but it is also one of the capabilities most easily obscured by marketing figures.

Being able to fit a 200-page financial report into the context window does not mean the model can accurately identify a critical assumption on page 180. Being able to process 100 hours of video does not mean it can consistently maintain relationships among people, events, and timelines. For agents, the real challenge is reintegrating information from long contexts into the decision-making process, not merely completing the act of “reading it in.”

Based on currently available public results, Qwen3.8’s progress appears to be concentrated precisely in this area: the model can not only process long documents and lengthy videos, but also use the information they contain for subsequent searches, analysis, code generation, and result verification.

Long-Horizon Task Capabilities Are Beginning to Create a Generational Gap

A set of internal and public evaluation results disclosed by the Qwen team helps explain why Qwen3.8-Max was able to top the Agentic Index:

| Benchmark | Qwen3.8-Max Score | Primary Focus | |---|---:|---| | PaperBench | 93.0 | Paper reproduction, code implementation, and experiment execution | | WideSearch | 81.9 | Broad information retrieval and multi-source integration | | Agent's Last Exam | 52.4 | Complex general-purpose agent tasks | | IF Bench | 82.8 | Instruction following | | GPQA Diamond | 92.6 | Advanced scientific reasoning | | BabyVision | 82.0 | Tool-free visual reasoning | | OSWorld-Verified | 86.1 | Computer use and graphical user interface tasks |

Its PaperBench score improved by 28.2 points over the previous generation, a more convincing signal than simply noting “how much larger the model has become.” Reproducing a paper is not a single-turn code-generation task. It usually requires the model to understand the paper, set up an environment, implement the algorithm, run experiments, and repeatedly make changes based on failure logs. If any one of these stages goes off track, the overall task cannot be completed.

The OSWorld-Verified score of 86.1 also deserves developers’ attention. Computer-use tasks require models to recognize interface states, understand the relationships among buttons, input fields, and menus, and adjust their actions based on changes to the page. These capabilities directly determine whether a model can be applied to browser automation, desktop productivity, enterprise software operation, and similar scenarios.

However, some of the specific scores above were disclosed by the model team and are not based on the same evaluations as Artificial Analysis’s third-party leaderboard. Vendor-reported data should still be treated with basic caution: the testing environment, tool permissions, inference budget, and number of retries can all significantly affect agent performance.

The Hardest Part for an Agent Is Not the First Step, but the Two-Hundredth

The core capability emphasized by Qwen3.8-Max is its ability to form an “execution–feedback–iteration” loop in long-running tasks.

That may sound like standard model-launch rhetoric, but in a real development environment, the difference is highly concrete.

For example, asking a model to “build a data dashboard” is not difficult. What is truly difficult is having it independently complete the following process: analyze requirements, design the page, create the project, install dependencies, write front-end and back-end code, start the service, inspect the page, discover misaligned charts, identify the CSS issue, rebuild the project, and finally verify that the filters and export functions work correctly.

Traditional coding models generally perform well in the first few steps, but begin losing track of state as the task grows longer. A more capable agent model must act like a patient engineer, continually comparing the current result against the original objective.

To evaluate this, the Qwen team developed RecreationBench. The model is given no source code and is not allowed to access the internet. It can only interact with an existing application, observe its interface and functional feedback, and then recreate the application from scratch. This task simultaneously requires visual understanding, code generation, interface operation, and iterative repair. No matter how strong the model is in any single capability, the final result will collapse if one part of the loop is significantly weaker than the others.

The challenges become even greater in longer commercial tasks. Legal document retrieval requires the model to maintain consistent standards across hundreds of documents; video analysis requires it to track actions across thousands of clips; and quantitative research requires it to continuously retrieve data, generate strategies, backtest them, and adjust parameters. Such tasks do not provide the model with a standard answer. Instead, they require it to establish its own acceptance criteria.

Qwen3.8-Max’s breakthrough lies in moving agents one step further from “capable of giving a demo” toward “capable of delivering results.”

For Developers, Topping the Leaderboard Does Not Mean Existing Models Can Be Replaced Directly

Looking only at the leaderboard, the simplest conclusion would be to replace Claude or GPT with Qwen. But real-world agent systems are not that simple.

Model capability is only one layer. Production performance is also affected by whether tool definitions are clear, how context is compressed, how failures are retried, how permission boundaries are configured, and who validates the results once a task is complete. Even a model ranked No. 1 on the Agentic Index may repeatedly call the same tool or waste large numbers of tokens pursuing the wrong approach if placed in a poorly designed workflow.

When evaluating Qwen3.8-Max, developers should pay closer attention to the following questions:

1. Long-task success rate rather than single-turn accuracy. Run end-to-end tests using real internal company tasks and record the final delivery success rate.
2. Cost of failure. Measure how many tool calls and tokens the model consumes when it fails, and whether it performs any irreversible actions.
3. Recoverability. Determine whether it can resume from a checkpoint after an interruption or must rerun the entire task.
4. Latency and throughput. Activating 95 billion parameters improves efficiency, but long-chain reasoning may still continue for minutes or even hours.
5. Sensitivity to tool descriptions. The same tool definitions often produce very different invocation behavior across models, so prompts cannot be migrated mechanically.
6. Security boundaries. Additional approval and sandboxing mechanisms are required when tasks involve code execution, file deletion, email sending, or writing to business systems.

Long context deserves particular attention. A 1-million-token context window is better used to raise the upper limit of task capacity; it should not become an excuse to skip retrieval, summarization, or memory management. Feeding all logs, documents, and conversation history into the model at once is generally both expensive and likely to introduce substantial irrelevant information.

A more reliable approach is still to manage context in layers: preserve recent operations in their original form, compress historical processes into state summaries, store key facts in structured memory, and retrieve source materials on demand.

The Pricing Advantage May Be Harder to Defend Against Than the Leaderboard Advantage

Artificial Analysis’s Intelligence Index gives Qwen3.8-Max a score of 56, above the median score of 32 for comparable models. Its publicly listed price is $2 per million input tokens.

Input pricing is especially important for agent products. An ordinary chat interaction may invoke the model only once, while an agent repeatedly carries context through planning, tool use, result checking, and failure retries. It is not uncommon for a single task to trigger dozens of inference calls, meaning differences in per-token pricing are multiplied across the entire call chain.

The real competitive pressure from Qwen3.8-Max may therefore come not merely from whom it surpassed with a score of 55.4, but from its combination of top-tier agent capabilities, a long context window, and more aggressive pricing.

Claude still has advantages in code quality, stability when following complex instructions, and a mature agent ecosystem. The GPT family, meanwhile, has a comprehensive tool platform, a broad developer base, and strong product distribution. What Qwen has now added is the one area that was previously the hardest to compensate for through lower pricing: the ability to complete long-horizon tasks themselves.

If subsequent third-party retesting can reliably reproduce these results, and if Qwen maintains its lead across different agent frameworks and tool budgets, Claude and GPT will no longer be facing merely a cheaper alternative, but a competitor capable of challenging them for the position of primary model.

One No. 1 Ranking Is Not Enough to Define a New Order

It must be recognized that agent leaderboards are more susceptible to environmental factors than traditional knowledge tests.

Browser versions, webpage changes, tool response times, maximum execution steps, and whether models are allowed to retry can all change the final results. Ranking first today does not mean ranking first across every type of task. A score of 55.4 also indicates that even the strongest current models remain far from reliably solving complex, open-ended tasks.

Moreover, the stronger an agent’s capabilities become, the greater the risks may be rather than the lower. When a model that only answers questions makes a mistake, it usually generates incorrect text. When a model capable of operating computers, executing code, and calling enterprise systems makes a mistake, it may directly modify data or trigger external actions.

A more reasonable interpretation of this result is therefore not that “Chinese models have comprehensively surpassed overseas models,” but that a Chinese model has taken the lead in a third-party comprehensive agent evaluation for the first time. It demonstrates that model competition is shifting away from parameters, context windows, and static benchmark scores toward the genuinely difficult challenge of task execution.

For Alibaba, three things will matter more than retaining the top ranking: reliably translating leaderboard performance into API performance that can be reproduced across different agent frameworks; controlling inference costs and latency for long-chain tasks; and establishing sufficiently transparent systems for security, logging, and permissions.

Leaderboards determine attention; production environments determine orders. Qwen3.8-Max has won the first contest, but the second is only just beginning.

References

• Official Qwen Model Page (Hugging Face): The entry point for models, model cards, and related open-source assets from the Qwen team, where public technical information about the Qwen family can be found.
• Official QwenLM Organization (GitHub): The official repository hub for Qwen code, toolchains, and open-source projects.