Apple has buried CoreML, and CoreAI is taking over to bet on on-device large models.

Apple sent CoreML to the museum at WWDC 2026.

On June 10, Cupertino officially released the CoreAI inference engine, replacing the CoreML framework, which had been in service for a full nine years. This is not a routine version number update, but a complete overhaul of the underlying architecture—CoreML was originally designed for small, static tasks like image classification and facial recognition in iOS 11. The compute assumptions from 2017 now look archaic in the era of large language models in 2026. CoreAI’s sole goal: make large models run efficiently and quickly on Apple Silicon devices.

From the first batch of third-party benchmarks, Apple didn’t submit a blank sheet, but it’s also not an overwhelming win.

Small Model Scenario: CoreAI Pinning MLX to the Ground

Let’s start with the most impressive data. On an M4-chip Mac running Qwen3 0.6B, CoreAI’s decoding speed is approximately 2.47× that of MLX. Switching to the iPhone 17 Pro, the advantage narrows to 1.6×, but remains significant.

The decoding speed metric (tok/s) requires no explanation for developers—it directly determines how long a user has to wait after pressing Enter for text to appear, word by word. Models of the 0.6B scale are typically used in on-device scenarios such as text completion, email summarization, and local search intent understanding—tasks that are latency-sensitive and frequently invoked. CoreAI holding a >2× lead at this scale means the same product experience could cut battery usage in half, or allow an older M1-era device to run it smoothly.

A quick note on MLX’s positioning: MLX was launched by Apple in late 2023 as a machine learning framework, but from the start was more geared toward research and training. The developer community running large models locally on MLX is a bit like “borrowing someone’s car for the highway.” CoreAI was born specifically for inference, with end-to-end optimization for Apple’s unified memory architecture, Metal backend, and ANE Neural Engine. Beating MLX on engineering grounds is within expectations.

Large Model Scenario: Advantage Quickly Shrinks

But the story changes at the 8B scale.

On an M4 Max running Qwen3 8B, CoreAI is only 5% faster than MLX. Statistically, this is almost negligible—any background process could easily cancel out that advantage.

What does this imply? It suggests CoreAI’s current optimization benefits are concentrated in small model paths, likely tied to targeted improvements for dynamic shapes, KV cache reuse, and small-batch inference. Once models scale up, bandwidth and compute limits of matrix multiplication become bottlenecks; there’s less room for framework-level optimization, and everyone’s playing at the limits of the underlying hardware.

This draws a clear boundary for Apple’s product strategy: Apple’s plan is to turn models in the 0.5B–3B range into system-level capabilities—local models for Apple Intelligence, writing tools, and Siri’s local inference path—all in CoreAI’s sweet spot. Developers wanting to run 8B, 14B, or larger models on Mac won’t lose much by sticking to MLX.

ANE or GPU? iPhone 17 Pro Shows Old Thermal Issues

An interesting set of data comes from sustained load tests. On the iPhone 17 Pro, long-running inference triggers thermal throttling faster on the GPU path, causing throughput to drop notably. The old CoreML + ANE (Apple Neural Engine) combination actually surpasses CoreAI in maintaining performance.

This is an old but repeatedly overlooked issue: when running large models on phones, peak performance doesn’t matter—sustained performance does. If a user asks a long question, and the model generates 30 seconds of text—with the first 5 seconds lightning-fast and the next 25 seconds painfully slow—the experience is worse than steady mid-speed output from start to finish.

Why is the ANE path more stable? Because it’s a fixed-point, low-precision compute unit with a lower power ceiling and less heat per unit of compute; the GPU uses a floating-point, high-throughput route—runs fast but heats quickly. CoreAI currently prioritizes GPU for throughput, but in mobile, heat-sensitive contexts, it may need to reincorporate ANE into scheduling, creating a hybrid strategy like “short tasks on GPU, long tasks falling back to ANE.”

Horizontal Comparison: Google’s LiteRT-LM is the Real Hidden Champion

If we only look at Apple’s internal competition, CoreAI delivers. But when placed in the industry context, the problems appear.

Google’s LiteRT-LM runs the Gemma model on an iPhone 17 Pro at 55.4 tokens/sec with only 641 MB memory usage. In comparison, Apple’s MLX running an equivalent load uses 2900 MB—4.5× more.

This is quite an embarrassing comparison: a cross-platform framework runs more memory-efficiently on Apple’s own hardware than Apple’s own toolchain. The reason is straightforward—LiteRT-LM is a deeply customized “model-specific engine” for the Gemma series, with quantization strategies, operator fusion, and KV cache layout tailored specifically for Gemma. CoreAI and MLX take a general-purpose approach that supports any model architecture, inevitably trading off extreme performance for universality.

The subtext here: the next battleground for on-device AI isn’t general frameworks, but vertical model–engine integration optimization. MiniCPM4 from Facewall Intelligence uses its in-house CPM.cu framework with InfLLMv2 sparse attention to accelerate long-text inference by 5×; Xiaomi’s XiaoAi team achieves 7–10× decoding speed boost on-device via speculative inference. These aren’t general-purpose solutions, but they’re far faster than general frameworks.

If Apple truly wants to lead on on-device AI, beyond CoreAI, it might need to create a deeply bound combination of “Apple’s own models + CoreAI”—similar to the tight coupling of Gemini Nano on Google Pixel.

Developer Perspective: Is CoreAI Worth Migrating To?

For developers working on on-device AI applications, several impact points of this migration are worth highlighting:

• More flexible model formats: CoreAI loosens model format requirements; the painful “must convert to .mlmodel” workflow of the CoreML era will likely be simplified.
• Higher memory limits: CoreML’s early support for memory mapping large models was limited; CoreAI clearly targets 7B–13B models here.
• MLX won’t be deprecated: Apple explicitly announced MLX will remain as the research/training toolchain, with CoreAI focused on inference deployment. This dual system is akin to PyTorch for training and ONNX Runtime for deployment.
• Biggest gains for small models: If your product is under 1B scale and latency-sensitive, migrating to CoreAI will probably yield >1.6× performance gains immediately. For models 8B and above, migration makes little difference.

A Restrained Step Forward

Frankly, CoreAI is not revolutionary—it’s catching up. Apple’s on-device AI strategy in 2024–2025 was widely criticized as “much noise, little action”—Apple Intelligence delays, Siri upgrades postponed, third-party developers lacking adequate tools. At least CoreAI pays off part of the developer-side toolchain debt.

But to claim Apple “redefines on-device inference” would be an overstatement. The 2.47× small-model advantage is a victory over its own MLX, not the industry’s best solution; being surpassed under sustained load by its own older CoreML+ANE setup further illustrates GPU’s real performance ceiling on mobile devices.

The real test for Apple’s on-device AI will come after iOS 19’s launch—whether system-level AI functions running on CoreAI can make ordinary users think, “Oh, my iPhone really got smarter.” Framework performance is impressive, but ultimately the product experience decides, and users vote with their feet.

On a related note, for developers wanting to deploy models like Qwen3 or Gemma on-device, it’s worth first running business logic and prototyping via cloud APIs. OpenAI Hub (openai-hub.com) supports the OpenAI format and lets you call GPT, Claude, Gemini, DeepSeek, and Qwen series with a single key, directly accessible in China without proxy hassles—saving considerable time in early comparison and selection. On-device/cloud cooperation is bound to be the norm going forward.

References

• Apple CoreAI On-Device AI Architecture Test: Qwen3 0.6B Decoding Speed 2.47× MLX on M4 Mac - IT Home — IT Home’s detailed coverage of WWDC 2026 CoreAI release and first-batch benchmark results