Moore Threads open-sources MusaCoder: Domestic GPU achieves 93.2% on KernelBench

On June 10, Moore Threads accomplished something not so easy: they released and open-sourced MusaCoder — a code large model specifically designed for “writing GPU kernels.” Two sizes, 9B and 27B, with weights already available on Hugging Face. The paper is simultaneously posted on arXiv.

The real highlight here isn’t just “another open-source code model,” but two key details: First, MusaCoder’s entire downstream training pipeline was run on the Quaozi intelligent computing cluster based on MTT S5000, from SFT, rejection sampling fine-tuning, to asynchronous rollout for reinforcement learning, online compilation/execution, reward calculation — the whole cycle stayed within the domestic GPU ecosystem; second, in KernelBench, a recognized hardcore kernel generation benchmark, the 27B RL version outperformed Claude Opus 4.7, DeepSeek-V4 Pro, GLM-5.1, and Kimi K2.6 across the board.

This is the first time a domestic GPU has independently delivered a leading specialized code model within the “train–validate–evaluate” full loop.

Not a General Code Model — Built for GPU Kernels

Let’s clarify its positioning. MusaCoder isn’t another general-purpose coding assistant competing with Claude Code or Cursor. Its target scenario is extremely narrow and hardcore: automatically generating high-performance CUDA / MUSA native kernel code from standard PyTorch operators.

Anyone who’s worked on GPU kernels knows the pain. A kernel isn’t like a regular function — writing something that “runs” is only step one:

• Parallel threads must be organized correctly
• Access patterns for shared memory, registers, and global memory must be efficient
• Index mapping cannot be misaligned
• Numerical precision must pass verification
• And in the end, it needs to actually be faster than the PyTorch baseline

General code models tend to fail here. They can beautifully complete a Python snippet, but when asked to write a fused operator’s CUDA kernel, most often they either fail to compile, compile but run incorrectly, or run correctly but slower than torch.compile — which is essentially useless.

That’s MusaCoder’s entry point. Moore Threads defines it as a specialized model for “low-level kernel generation tasks,” paired with a complete GPU-native kernel-oriented downstream training methodology.

Training Methodology: From “Can Run” to “Runs Correctly and Fast”

In the paper, this process is divided into three stages: supervised warm-up, task alignment, and execution-feedback-driven RL. Sounds like a standard recipe, but the devil is in the details.

Data Construction: Explicitly Inject Shape Information

The easiest place for kernel generation to fail is tensor shapes and index relationships. During data preparation, MusaCoder does two things: first, structured reasoning process, making the model infer shapes before writing code; second, explicit shape injection, forcing tensor shapes and memory layouts — which normally the model would figure out itself — into the context.

This injected prior knowledge aims to solve the cold-start problem when transferring from general coding ability to kernel generation — you can’t expect a model to learn stride and offset just from looking at .cu files on GitHub.

MooreEval: Distributed Execution Verification

This is the key infrastructure that makes the whole training pipeline work.

MooreEval is Moore Threads’ own distributed execution verification system, handling four tasks:

1. Automatic compilation
2. Execution + correctness verification
3. Performance benchmarking (vs. PyTorch Eager and torch.compile)
4. Anti-cheating detection

The last one is especially notable. In code generation RL training, models can easily “cheat” — e.g., directly calling underlying libraries to bypass implementation, or providing a degenerate version to fool the reward. MooreEval blocks these paths, forcing the model to honestly write kernels.

RL Stage: PrimeEcho, MirrorPop, BDR

Reinforcement learning is MusaCoder’s most interesting part. GPU kernel tasks are notoriously unfriendly to RL training:

• Multi-round repair scenarios produce sparse signals
• Long-tail hard samples cause all rollouts to fail, gradients to be zero
• Training stability is poor

Moore Threads introduced three mechanisms:

• PrimeEcho: multi-round rewards for first-round anchoring, ensuring multi-round debugging feedback stays consistent
• Buffered Dynamic Retry (BDR): solves “all failure” hard samples — caching samples and retrying repeatedly to extract training signals
• MirrorPop: off-policy sequence filtering for stable GRPO training

This combination creates a complete loop from SFT → RFT → execution-feedback RL.

Benchmark: Outperforming Closed-Source Flagships

Let’s look directly at KernelBench data. The evaluation metric is Pass@8 (at least 1 valid result in 8 samples) and Avg.@8 (average correctness over 8 samples):

| Model | Overall Pass@8 | Overall Avg.@8 | |---|---|---| | MusaCoder-27B-RL | 93.2% | 88.60% | | Claude Opus 4.7 | 87.2% | 77.30% | | DeepSeek-V4 Pro | — | — | | GLM-5.1 | — | — | | Kimi K2.6 | — | — |

That’s 6 points higher Pass@8 and 11 points higher Avg.@8 compared to Claude Opus 4.7.

More importantly, Level 3 tasks — the hardest tier in KernelBench, involving complex shape inference, index mapping, and multi-operator combinations. On these, MusaCoder-27B-RL’s Pass@8 and Avg.@8 lead Claude Opus 4.7 by 18 percentage points and 26.5 percentage points, respectively.

Such a lead can’t be explained by “tuned parameters” — it’s a structural advantage of a task-specific model over a general-purpose model.

Another tougher metric — Faster Rate: a generated kernel must pass correctness and legality checks and run faster than the PyTorch baseline to be counted:

• MusaCoder-27B-RL: vs. PyTorch Eager 15.0%, vs. torch.compile 9.2%
• Claude Opus 4.7: 11.8% / 7.5%

This shows MusaCoder doesn’t just produce correct code — its output actually accelerates execution, which is what kernel engineers truly care about.

Domestic GPU Sustaining the “Full Training Cycle”

Beyond the model itself, the most significant signal is: the entire downstream training process used no NVIDIA GPUs.

SFT, RFT, RL, asynchronous rollout, online compilation/execution verification, reward calculation — all ran on the Quaozi cluster built with MTT S5000 GPUs.

It may sound routine, but for domestic GPUs it was a real stress test. Kernel generation training is far more demanding than typical large model fine-tuning: every rollout triggers code generation, compilation, execution, performance measurement, feedback write-back. Hardware, compiler stack, runtime, scheduling, and evaluation infrastructure requirements overlap — if any link fails, the RL pipeline collapses.

Previously, domestic GPUs were often criticized as “good for inference, marginal for fine-tuning, fail on complex training.” MusaCoder publicly demonstrates: domestic GPUs can sustain a complete reinforcement learning training flow with online execution feedback.

Combined with Moore Threads’ previously public cluster data — Dense model MFU 60%, MoE 40%, effective training time 90%, linear scaling efficiency of 95% across 8,000 GPUs — this pathway is taking shape.

What This Means for Developers

Practically speaking, MusaCoder is built for two types of people:

First, GPU high-performance computing engineers. Handwriting CUDA/MUSA kernels has always been high-barrier and cognitively taxing, and AI-assisted kernel-writing tools have been lacking. MusaCoder’s positioning exactly fills this gap — give it a PyTorch reference implementation, it outputs kernel code, paired with MooreEval for execution verification, the loop completes.

Second, universities and research institutes working on heterogeneous computing. Moore Threads open-sourced the whole package (including 9B, 27B weights and paper), effectively providing an open experimental platform for directions like AI compilation optimization and automated kernel generation.

For the MUSA ecosystem, this is a repositioning. Previously MUSA SDK 5.1.0 matched CUDA 12.8, emphasizing “100% compatibility with all PyTorch operators” — essentially a follower. MusaCoder changes the narrative: from “I can do whatever CUDA can” to “I have a specialized kernel-generation model that CUDA doesn’t.”

This is MUSA’s first differentiated experience where “doing something is easier than on CUDA.” Whether it snowballs depends on follow-up — IDE plugins, profiling tools, automated debugging toolchains — Moore Threads says these are already on the roadmap.

A Sober Judgment

A few reservations worth keeping:

• Benchmark limitations — KernelBench is public, but still a single benchmark. MusaCoder-27B-RL’s big advantage on Level 3 may partly come from training data distribution covering evaluation tasks. Real-world kernels are far more complex.
• 27B deployment cost — specialized kernel-writing at 27B isn’t friendly to small teams. The gap between the 9B and 27B versions will determine adoption.
• Actual demand for MUSA kernels — the model is useful for CUDA developers too, but Moore Threads surely hopes it drives MUSA kernel creation. MUSA’s developer base is an order of magnitude smaller than CUDA’s; how much the model can feed back into the ecosystem remains to be seen.

But setting these aside, MusaCoder is solid work. Moving a domestic GPU company from “selling cards” to “selling cards + training a SOTA specialized model” — at least in the AI Infra niche — Moore Threads has closed the ecosystem loop.

The model is already live on Hugging Face. Developers interested in kernel generation can pull it and try.

For open-source models, OpenAI Hub is also continuously integrating mainstream open-source code models, making it easy for developers to compare and test with a unified interface. If desired, MusaCoder can be tested alongside Claude, DeepSeek, and others on actual tasks.

References

• ITHome: Moore Threads open-sources MusaCoder — first code large model trained on domestic full-function GPU full stack — Main report on MusaCoder release and open source, includes KernelBench evaluation data
• Hugging Face: MooreThreads/MusaCoder-27B — MusaCoder 27B model weights and card info