华为开源 openPangu-2.0 预训练、SFT与RL训练代码

Huawei open-sourced the pretraining, SFT, and post-training RL code for openPangu-2.0 today, filling in the training components beyond the model weights and inference modules. For developers, this makes training practices on Ascend more visible, but open-sourcing the code does not mean the barriers to training have disappeared.

Huawei Open-Sources Pangu 2.0 Training Code

Huawei announced today that the pre-training, supervised fine-tuning (SFT), and post-training reinforcement learning (RL) code for openPangu-2.0 is now officially available as open source. The code is split between two repositories, one for training and the other for RL, and has been released through the Ascend Tribe community.

The focus of this update is not simply the release of another set of model weights, but the opening up of the key code paths spanning pre-training through post-training. For teams looking to reproduce or modify large model training workflows on Ascend hardware, this provides greater engineering value than downloading model weights alone.

Diagram of the openPangu-2.0 training workflow, showing the three stages of pre-training, SFT, and reinforcement learning, as well as the Ascend computing platform

From Open Weights to an Open Training Pipeline

openPangu is Huawei's open-source AI model brand. One of its core goals is to provide practical references based on native Ascend training and inference technologies. Components of openPangu-2.0 had already been released in stages: the Flash model was open-sourced in June this year, while the Pro model and technical report were released in July. Huawei also said at the time that model-related components would be opened up in batches. Today's release of the pre-training, SFT, and RL code further expands the scope from model use and inference to the training process.

These categories of code serve different purposes. Pre-training enables a model to learn general language and knowledge capabilities from large-scale corpora. SFT uses curated instructions and examples to teach the model how to respond as required. RL then continues to adjust the model's behavior through feedback or task objectives. They can be understood as three consecutive stages of model training: establishing a foundation, teaching the rules, and then reinforcing performance.

Successfully running this entire pipeline depends on a range of factors, including data processing, parallelization strategies, operator implementations, training frameworks, and hardware configurations. Releasing the training code alone does not necessarily mean that anyone can reproduce large-scale training with a single command. However, the code at least gives external developers an opportunity to examine key implementations, understand Huawei's engineering choices, and conduct adaptation and experimentation in comparable Ascend environments.

Open Source Has More Practical Significance for the Ascend Ecosystem

Competition in the hardware ecosystem for large model training is not just about chip computing power. Framework compatibility, operator maturity, the stability of parallel training, and the ability to diagnose problems are equally important in determining whether developers can make effective use of the hardware. With the training code now open, external teams can examine these engineering aspects beyond the model itself and assess whether the existing toolchain is suitable for their tasks.

This is the most noteworthy aspect of the release: it moves training practices on Ascend from vendor descriptions to code that can be inspected by the community. For companies or research teams evaluating domestic computing infrastructure, readable, modifiable, and testable implementations provide information that is more relevant to real-world decisions than peak performance specifications alone. Developers can further examine code dependencies, parallel configurations, and training workflows before estimating the adaptation costs required to migrate existing workloads.

However, the value of open-source code depends on whether it can actually be used, not merely whether it has been uploaded. Large-scale model training typically requires a complete hardware setup, a compatible software stack, sufficient data, and operational expertise. Opening the repositories does not automatically eliminate these barriers. If the documentation, configuration examples, dependency versions, and issue-reporting mechanisms are incomplete, external teams may still need to invest considerable time to reproduce the results. For ecosystem development, ongoing compatibility maintenance and community support are just as important as the initial code release.

Model Scale Does Not Equal Code Performance

Documentation for the Flash version of openPangu-2.0 indicates that it is a mixture-of-experts (MoE) model trained on Ascend NPUs, with approximately 92 billion parameters, around 6 billion activated parameters per token, and support for a 512K context window. The documentation also states that it was trained on approximately 34 trillion tokens. Its post-training stage involved SFT, multiple specialized RL tasks, online distillation, and other work. This information can help developers understand the model context for which the open-source training code was designed, but it should not be treated as a direct performance assessment of the newly released code.

Model parameter count, training data volume, and context length describe different dimensions. Parameter count reflects part of a model's capacity, activated parameters relate to MoE inference computation, and long context concerns the model's ability to process lengthy inputs. None of these metrics alone demonstrates the efficiency or reproducibility of the training code, or the model's effectiveness in specific business scenarios. Evaluating this open-source release also requires examining which modules the public repositories actually provide, how they correspond to hardware and software versions, and whether the community can reproduce experiments and propose improvements based on them.

What This Means for Developers

For teams already using Ascend, the training code can help map out the engineering path from pre-training to post-training and serve as a reference for migration, parameter tuning, or internal solution evaluation. For researchers, it provides an entry point for examining the complete training methodology and implementation details. For model application teams, its short-term value may be less apparent than direct access to model weights or inference APIs. In the long term, however, transparency across the training pipeline helps determine whether the model and underlying platform are capable of sustained iteration.

It is important to distinguish between open-sourcing training code and releasing training data. Nor does this mean that the implementation provided in the repositories can be used without modification for any model or hardware platform. Before using the code, developers should still verify the repositories' licenses, dependency environments, hardware requirements, and the intended scope of each component. Model licenses and code licenses may differ in particular, so the corresponding files should be reviewed separately before downloading or using them commercially.

From an industry perspective, Huawei's move makes the open-source story of openPangu-2.0 more complete. By successively releasing model weights, inference-related components, and training code, Huawei has covered multiple stages from model use to training and development. Whether this will further reduce the trial-and-error costs of training on Ascend will depend on the quality of the code, the completeness of the documentation, and the progress of subsequent community collaboration. For now, the most reasonable conclusion is that this release gives developers greater opportunities to inspect and participate in Ascend training practices, but its ecosystem impact will still need to be validated through real-world projects.

Code Repositories

The code repositories announced by Huawei are divided into two parts: one provides code related to openPangu-2.0 training, while the other provides post-training RL code. Because the official repositories are hosted on GitCode, links to that site are not included here. Developers can search the Ascend Tribe community using the repository names openPangu-2.0-Training and openPangu-2.0-RL. Before use, it is recommended to first read the repository documentation, license, and environment configuration instructions, and to confirm the corresponding model version and Ascend software stack requirements.