openEuler 24.03 LTS SP4: Another Evolution of the AI-Native Operating System

Yesterday, openEuler 24.03 LTS SP4 made its official debut at the Open Atom Open Source Ecology Conference in Beijing. This is the fourth service pack version in the openEuler 24.03 LTS series, released about three months after the previous SP3 version.

To be honest, operating system version updates typically don’t attract widespread attention in developer circles—most of the time, it’s just fixes and patches. But this SP4 release is a bit different: its upgrades in AI infrastructure support are quite substantial, especially in AI graph compiler and inference scenario optimization, making it worth a closer look.

Based on Linux 6.6 Kernel: More Than Just a Version Number Upgrade

openEuler 24.03 LTS SP4 continues to build on the Linux 6.6 kernel. This choice says a lot.

Linux 6.6, a long-term support version released at the end of 2023, has notable improvements to core subsystems such as the scheduler, memory management, and file systems over previous LTS kernels. The openEuler community chose not to chase newer versions like 6.7 or higher, instead polishing the stable 6.6 baseline—an approach that is pragmatic for enterprise users.

Key Kernel-Level Enhancements

Elastic Memory Management is one of the highlights of this upgrade. In cloud computing and container deployment scenarios, dynamically allocating memory resources has always been a tricky problem. The traditional approach either reserves too much memory (wasting resources) or too little (causing Out of Memory errors). The SP4 release introduces an elastic memory mechanism that adjusts allocation strategies based on actual workload, balancing application stability with higher resource utilization.

64K Kernel Page Support is another notable feature. By default, the Linux kernel uses a 4KB page size, which generates large page table overhead when handling massive datasets. 64K pages can significantly reduce the number of page table entries, giving tangible performance gains for memory-intensive AI training and inference tasks. According to community test data, memory access efficiency can improve by over 20% in specific scenarios.

However, 64K pages aren’t a silver bullet—they may cause memory waste due to internal fragmentation in scenarios involving frequent small file reads/writes. So openEuler provides flexible configuration options for users to choose the appropriate page size based on workload.

AI Graph Compiler: From "Usable" to "User-Friendly"

The most interesting upgrade for me in SP4 is the AI graph compiler.

What Is an AI Graph Compiler?

Here’s some context. Modern deep learning frameworks (PyTorch, TensorFlow, etc.) execute neural networks by first converting the model into a computation graph, then executing it. An AI graph compiler optimizes this computation graph to run faster on specific hardware.

You can think of it as the AI equivalent of a traditional compiler. GCC or Clang optimize C code into machine code with techniques like loop unrolling and constant folding; an AI graph compiler similarly optimizes neural network computation graphs.

What’s New in SP4’s Graph Compiler?

According to official information, SP4’s AI graph compiler enhancements focus on:

1. Operator Fusion Optimization

Operator fusion merges consecutive small operations into a single larger operation to reduce memory reads/writes of intermediate results. SP4 expands the operator fusion pattern library to identify and optimize more operator combination patterns.

For example, in Transformer architectures, the common “MatMul + Add + LayerNorm” sequence can be fused into a single CUDA kernel (on NVIDIA GPUs) or a single NPU operator (on Ascend AI accelerators), eliminating two unnecessary memory transfers.

2. Heterogeneous Compute Scheduling

AI servers now typically have multiple compute units: CPU, GPU, NPU, even FPGA. Assigning tasks across these heterogeneous devices is complex.

SP4’s graph compiler integrates smarter scheduling strategies—automatically deciding which operators should run on GPUs, NPUs, or CPUs based on computational characteristics and hardware capabilities. Previously, developers had to manually configure this; now, the system handles most of it.

3. Memory Reuse Optimization

VRAM usage during large model inference has long been a pain point. SP4 introduces more aggressive memory reuse during graph compilation, analyzing dependencies to let different intermediate tensors share the same memory space whenever possible.

For example, in a dependency chain A → B → C → D, once C starts computing, A’s memory can be reclaimed and reused. SP4’s compiler automatically identifies such patterns and generates corresponding memory management code.

Inference Scenario Optimization: Not Just Training

In recent years, AI focus has mostly been on training—how to train larger models with more data. Starting in 2024, inference has gained importance. Training is one-off (or infrequent), while inference is continuous, directly affecting operational costs and user experience.

What Has SP4 Done on Inference?

Comprehensive Software Stack Updates

openEuler 24.03 LTS SP4 systematically updates support for mainstream AI inference frameworks:

• CUDA/cuDNN: Supports the latest stable versions, optimized for NVIDIA’s Hopper architecture (H100, etc.)
• CANN (Huawei Ascend computing architecture): Deep integration with dedicated optimization for Ascend 910 NPUs
• oneAPI/OpenVINO: Full support for Intel’s AI software stack, including CPU and GPU inference
• vLLM: Worth noting—vLLM is one of the most popular large-model inference frameworks now. The openEuler and vLLM communities collaborate closely, making vLLM deployment on openEuler ready out-of-the-box.

Low-Latency Communication Optimization

In distributed inference scenarios, communication latency between nodes is critical for performance. SP4 further optimizes RDMA and NVLink high-speed interconnects, reducing software overhead for transfers.

Community test data shows that in an 8-card parallel inference setup, node-to-node communication latency dropped by ~15%—a meaningful improvement for real-time AI applications such as chatbots or live translation.

Agent Sandbox

This new feature ties into the surge of AI Agents.

AI Agents that execute code, operate file systems, or call external APIs pose security risks. SP4’s Agent sandbox isolates execution environments so Agents run in restricted contexts, avoiding malicious or erroneous operations impacting the host.

Technically, the sandbox uses Linux namespaces and cgroups, with custom policies for AI Agent scenarios—useful for teams deploying AI Agents in production.

Virtualization and Cloud Computing: The Foundation Matters

Though AI is SP4’s main selling point, virtualization and cloud computing support remain vital for an enterprise OS.

Virtualization Enhancements

SP4 improves KVM virtualization in several ways:

1. Enhanced vCPU Hot-Plug

VM CPU core counts can now be adjusted dynamically with less performance jitter during hot-plugging—minimizing impact on running applications.

2. Memory Ballooning Improvements

SP4 improves balloon driver efficiency, reclaiming idle VM memory faster and with less impact on internal applications.

3. I/O Passthrough Performance Boost

For high-performance I/O scenarios (databases, storage services), passthrough of physical devices to VMs is common. SP4 optimizes the VFIO framework to reduce software overhead in device passthrough.

Container Runtime Support

In container space, SP4 updates containerd and CRI-O with the latest OCI spec support—streamlining Kubernetes deployment and management on openEuler.

Community also highlights support for Huawei’s “Lingqu Super Node”—an AI supercomputing node solution connecting multiple servers into a logical “super node.” openEuler adapts scheduling and resource management for this architecture.

Intelligent Operations and Maintenance: Self-Caring Systems

Operations and maintenance (O&M) is tough, especially for large clusters. SP4 makes notable progress in intelligent O&M.

Intelligent Tuning

Not a new concept, but SP4’s implementation is more mature—automatically adjusting kernel parameters and scheduling strategies based on runtime load, without manual intervention.

Example: when detecting memory-intensive workloads, the system adjusts page recycling and swap behavior; for CPU-intensive workloads, it tweaks time slice allocation. This adaptability is especially useful for mixed workloads.

Intelligent O&M Agent

Kylin Software showcased an intelligent O&M Agent based on openEuler. It can:

• Automatically analyze logs to identify potential issues
• Perform fault localization using a historical knowledge base
• Generate repair suggestions and even execute fixes automatically

The demo showed reliable diagnostics for common issues (disk space shortage, process crashes, network connectivity problems). Complex production issues still require human intervention.

A Small Step Toward Internationalization

Another noteworthy update: Kazakhstan’s Freedom Cloud announced joining the openEuler community as the first overseas ecological partner.

Internationalization has long been a challenge for domestic open source projects—technical capability is fine, but community management, documentation translation, and localization lag behind. Freedom Cloud’s addition is a small step, but shows that openEuler is seriously considering internationalization.

According to officials, Freedom Cloud will donate to the community, and openEuler’s tech and ecosystem experience will be exported to Central Asia. This two-way cooperation is healthier than purely one-way tech export.

Partner Practices

In the ecology sharing session, several partners showcased products and solutions based on openEuler.

KylinSec demonstrated server OS solutions for sectors like power, government, and special industries—fields with high demands for stability and security—adding security hardening and compliance layers to openEuler.

iSoftStone Tianhe AIOS aims to evolve the OS from a “resource management base” into an “AI application enablement platform”—embedding AI model management and inference scheduling capabilities into the OS layer for easier AI use in applications.

FusionOS 26 by xFusion focuses on practical pain points of enterprise AI deployment: compute allocation, token consumption metering, multi-version model management—issues enterprises must address for large-scale AI rollout.

What Does This Mean for Developers?

For ordinary developers, is openEuler 24.03 LTS SP4 worth attention?

If you’re doing AI-related development, SP4 is worth trying—especially for deploying models on Ascend NPUs, where openEuler + CANN is currently the most mature solution. Collaboration with vLLM means large-model inference will keep improving.

If you manage cloud or container platforms, SP4’s virtualization optimizations and intelligent tuning may ease your workload. The intelligent O&M Agent is still maturing but worth watching long-term.

If you just want a stable Linux server OS, openEuler 24.03 LTS is already a solid choice. As the fourth service pack, SP4 offers assured stability.

openEuler’s greatest strength remains its ecosystem—especially with domestic hardware. If you need to run on Kunpeng, Feitian CPUs, or Ascend NPUs, openEuler is essentially the only OS seriously supporting these platforms.

Final Thoughts

Since its open source release in 2019, openEuler has been around for over six years. From “Huawei’s internal OS made public” to “multi-vendor ecosystem co-building,” the change is significant.

The 24.03 LTS SP4 release offers real technical progress, especially in AI. But perhaps more importantly, there’s community evolution—Freedom Cloud joining, 10 developers receiving community thanks, multiple partner practice shares. Such “soft power” accumulation is often more critical for the long-term success of an open source project.

The path for domestic operating systems isn’t easy. Technology can catch up, but ecosystem building takes time. What openEuler is doing now is directionally right—it just needs more patience.

---

References:

1. openEuler 24.03 LTS SP4 Release: Elastic Memory, 64K Kernel, AI Graph Compiler Fully Upgraded - IT Home - IT Home report on the openEuler 24.03 LTS SP4 release

2. openEuler 24.03 LTS SP4 Release Details - IT Home Mobile Edition - Mobile report including core feature introduction