The 35B Model Runs on Phones: MiniCPM-V 4.7 Released

ModelBest today (October 13) released MiniCPM-V 4.7, its new-generation on-device multimodal large model, in two versions: 35B-A3B and 1B.

The 35B-A3B is the first model in the MiniCPM-V series to use a MoE (Mixture of Experts) architecture. ModelBest says it uses only about 3 GB of RAM on an iPhone 17 Pro Max, reaches decoding speeds of 13+ tokens/s, and supports smooth, lossless operation on-device.

The key point isn’t just that “a 35B-parameter model can run on a phone.” What’s really worth watching is that competition among on-device models is shifting from “whose model has fewer parameters” to “who can run a larger model at a lower real-world cost.”

35B Is the Total Parameter Count; Only About 3B Are Actually Used for Computation

First, let’s explain what 35B-A3B means.

35B represents a total model parameter count of about 35 billion, while A3B means that about 3 billion parameters are activated for each inference. Unlike traditional dense models, a MoE model doesn’t consist of just one complete network. Instead, it contains multiple “expert modules” responsible for different tasks. When processing an input, the model selects a subset of these experts to participate in the computation based on its content.

You can think of it as a company with 35 specialist teams, but each project only needs to call on three of them. The company’s overall capabilities depend on the expertise accumulated across all its teams, while the cost of executing an individual project is closer to that of engaging just three teams.

So, in terms of computation, MiniCPM-V 4.7 35B-A3B is closer to a 3B-scale model, but it isn’t the same as an ordinary 3B dense model. A larger pool of experts gives the model greater capacity and broader task coverage, while sparse activation reduces the computational load of each inference.

Of course, MoE doesn’t solve the problem entirely. The weights for every expert still need to be stored. The real challenge for on-device use is the limited memory and slower storage bandwidth. The model can’t simply keep all expert weights resident in memory; otherwise, the memory footprint of the full model could easily cancel out the computational advantage of “using only 3B parameters at a time.”

LES Addresses the Problem of “Expert Weights Not Loading in Time”

This release from ModelBest uses LES (Lookahead Expert Scheduling).

In traditional MoE inference, the model first determines which experts it will need next, then reads their corresponding weights from storage. On servers, this read latency can be mitigated with faster video memory, RAM, and parallel hardware. But on a phone, the speed gap between computation and storage is more pronounced. If expert weights aren’t available in time, inference can stall frequently.

LES works by predicting which experts may be needed in subsequent steps and beginning to load their weights while the current computation is still underway. In other words, it aims to run computation and data transfer in parallel rather than making them happen one after the other.

This is similar to preloading during video playback: while users are watching the current scene, the player has already cached what comes next. If its prediction is accurate, there’s no need to wait for the next segment to load when it’s time to play it.

According to information published by ModelBest, LES, combined with weight quantization, storage-layout optimization, and other techniques, can significantly reduce delays caused by switching between experts. The company reports an approximately 5× acceleration in on-device inference. More importantly, this method preserves the model’s original expert selection mechanism; it doesn’t gain speed simply by reducing the number of experts. That’s why ModelBest describes it as “lossless” operation.

However, “lossless” should be understood precisely. It mainly means that there is no significant sacrifice in the original model’s capabilities or expert-routing mechanism. It doesn’t mean that the same speed can be consistently achieved on every device, at every context length, and with every type of input. On-device inference is still affected by chip performance, memory bandwidth, flash storage speed, quantization methods, and system scheduling.

3 GB of RAM Isn’t the Full Size of a 35B Model

“Running a 35B model with 3 GB of RAM” is an easy headline to spread, but developers shouldn’t interpret it as “a 35B model only needs 3 GB of storage.”

The 3 GB figure is closer to the RAM footprint under a specific deployment configuration. It typically depends on a suite of engineering optimizations, including quantization, weight chunking, on-demand expert loading, and cache management. The full model weights still require more storage, and their exact size depends on the precision format and the model files being distributed.

In other words, MiniCPM-V 4.7 solves the problem of “how to make a model work with limited memory,” not how to make 35 billion parameters disappear. Developers deploying the model need to pay attention to all of the following metrics:

• Model file size: Determines the impact on the app’s installation package, initial download, and local storage.
• Peak RAM usage: Determines whether the model may trigger background app termination or a crash.
• Time to first token: Affects how natural interactions such as photo-based Q&A and real-time translation feel.
• Sustained decoding speed: Determines whether long-form output and ongoing conversations will stutter.
• Power consumption and heat: Determine whether the model can run for extended periods, rather than only completing a single demo.
• Stability across different chips: Determines whether the model can reach a wider range of devices beyond flagships.

ModelBest has so far reported test results of about 3 GB of RAM and 13+ tokens/s on an iPhone 17 Pro Max. That’s an attractive result for on-device multimodal inference, but it comes from a specific hardware and deployment configuration and can’t be directly extrapolated to all phones.

Support for iOS, Android, and HarmonyOS

In terms of operating system and hardware support, MiniCPM-V 4.7 supports iOS, Android, and HarmonyOS, and is adapted for chip platforms from Intel, AMD, Arm, Rockchip, Houmo, and others.

This is more valuable than simply releasing model weights. The barriers to on-device models aren’t limited to the model architecture itself; they also include compilation, operator support, memory management, thread scheduling, and compatibility with different inference frameworks. If a model can only run on a single chip or runtime, it still has a long way to go before it can make its way into real products.

Multi-platform support means the model can cover more local computing scenarios, such as:

1. Offline visual Q&A: Users can take photos of devices, receipts, whiteboards, or on-site scenes and get answers directly on-device.
2. On-device document understanding: Contracts, spreadsheets, and internal materials don’t need to be uploaded to the cloud, making this suitable for privacy-sensitive enterprise scenarios.
3. Real-time camera analysis: Video streams can be processed in poor or unavailable network conditions, reducing the latency caused by network round trips.
4. Local intelligent assistants: Text, images, and screen content can be processed by the model without routing all data through remote servers.
5. Robotics and embedded devices: Low-latency vision tasks can be performed on industrial PCs, development boards, and edge devices.

For developers, the most practical value of these models isn’t replacing cloud-based large models; it’s moving some tasks to the device—tasks that need low latency, can’t be performed online, or involve data that isn’t convenient to upload. Cloud models remain well suited to complex reasoning and high concurrency, while on-device models handle privacy, responsiveness, and offline availability.

The 1B Version Targets Lower-End Devices

In addition to 35B-A3B, MiniCPM-V 4.7 is also available in a 1B version.

The 1B model has a more straightforward positioning: it doesn’t aim to cover every complex task, but instead tries to bring visual understanding to more devices with limited memory and computing power. ModelBest says the new version improves on MiniCPM-V 4.6 in core capabilities, overall scores, and inference efficiency, and leads Qwen3.5 0.8B in the relevant comparisons.

The two versions have clearly differentiated roles:

• 35B-A3B: For flagship phones, tablets, edge workstations, and local applications with higher capability requirements. Its focus is greater model capacity and on-device efficiency.
• 1B: For low- and mid-power devices, real-time scenarios, and large-scale deployment. Its focus is low latency, low memory use, and lower hardware costs.

If an application only needs OCR, simple image Q&A, or visual classification in a fixed format, the 1B version may be more cost-effective than 35B-A3B. Conversely, if a task involves complex chart understanding, multi-turn visual conversations, or longer contexts, 35B-A3B has a chance to demonstrate the capacity advantage of MoE.

SRFT: More Than Just Weights Are Open-Sourced

ModelBest has also open-sourced its Self-Replay Fine-Tuning (SRFT) approach.

On-device models often gain real-world adoption not through general benchmarks, but through developers’ ability to adapt them to their own data and workflows. The problem is that traditional fine-tuning can easily result in “learning a new task but forgetting old capabilities”—a phenomenon known as catastrophic forgetting. This is especially common in specialized domains with limited data.

SRFT’s core approach is to preserve the model’s existing capabilities through a self-replay mechanism while it learns new data, reducing the cost of further training and the risk of forgetting. For developers, this means they can adapt MiniCPM-V 4.7 to specific tasks such as industrial quality inspection, receipt recognition, educational problem solving, and equipment maintenance at lower cost, without having to train a multimodal model from scratch.

This open-source release may be more practically valuable than another benchmark score. Model weights answer the question “Can it be deployed?” A fine-tuning approach answers “Can it be used in my business?”

When Comparing with the Qwen Series, Scores Aren’t the Only Thing That Matters

ModelBest says that in Instruct mode, the 35B-A3B version achieves a higher overall score than Qwen3.6 35B-A3B with less inference time; the 1B version leads Qwen3.5 0.8B in overall performance and inference efficiency.

These results are currently based mainly on comparisons published by the company. The conclusions still need to be retested by the community using the same hardware, quantization precision, context length, and real-world business data. This is especially true for on-device models, where benchmark scores and user experience are often separated by a whole set of engineering details: the same model can have different time-to-first-token, sustained speed, and peak memory usage on different inference backends.

But from a product strategy perspective, MiniCPM-V 4.7’s direction is clear: it isn’t simply chasing parameter scale, but combining optimizations across model capacity, sparse computation, weight loading, and chip adaptation. That’s also the biggest difference between it and many multimodal models that only provide server-side inference.

Assessment: A Key Validation of On-Device MoE, but Not the Finish Line

The most valuable thing about MiniCPM-V 4.7 is that it moves the question of whether a model with tens of billions of parameters can run on a phone from proof of concept to something people can actually try.

On-device large models have typically focused on sizes of 1B, 3B, or 7B parameters, compressed and quantized for practical reasons: a phone’s memory, thermal limits, and battery mean that models can’t grow indefinitely. As MoE, low-bit quantization, and expert scheduling mature, the total parameter count can continue to grow while the computation required for each inference remains manageable.

But this approach also has clear limits.

First, on-device MoE depends heavily on the storage system. If expert weights are frequently swapped in and out, flash bandwidth and access latency will directly determine the user experience. Second, 13+ tokens/s mainly reflects decoding speed; image encoding, video frame extraction, and time to first response for multimodal inputs are equally important. Third, power consumption and heat during sustained operation still need long-duration testing. Getting a model to run briefly doesn’t mean it can work continuously on a phone for hours.

So MiniCPM-V 4.7 is more like a clear technical signal: on-device models are no longer just “smaller versions” of cloud-based large models; they’re developing their own engineering approach. For developers, the real questions to test aren’t “Can 35B run?” but whether it can run reliably on the target device, whether its power consumption is acceptable, whether it can reduce cloud costs, and whether it offers irreplaceable value in terms of privacy and latency.

The model is open-source. Developers can choose the 35B-A3B or 1B version based on their device type and carry out on-device testing using the official inference and fine-tuning approaches. For applications that need multimodal capabilities but don’t want to send all their data to the cloud, MiniCPM-V 4.7 is worth adding to the test list. For scenarios that only need simple visual recognition, the 1B version may actually be the safer starting point.

Sources

• ITHome: ModelBest Releases Its New-Generation On-Device Multimodal Large Model, MiniCPM-V 4.7: Provides information on the release date, model specifications, on-device memory usage, decoding speed, and platform support.
• OpenBMB MiniCPM-V GitHub repository: Open-source model code, usage instructions, and related technical materials.