ModelBest and OpenBMB Open-Source MiniCPM-V 4.7, a 35B MoE That Runs Offline on Phones

ModelBest and OpenBMB have officially open-sourced MiniCPM-V 4.7, including 35B-A3B and 1B versions. Among them, 35B-A3B has 35B total parameters and activates only about 3B per inference, making it the first MoE model in the MiniCPM-V series; the 1B version actually has 1.3B parameters, focuses on extremely small size, and covers OCR, math, and video understanding.

Reports claim 35B-A3B needs only about 3G of memory to run smoothly on iOS and Android devices, and on iPhone 17 Pro Max, while taking up only about 3G of memory, decoding speed can reach over 13 tokens per second. These numbers come from the vendor and the reporting party's own accounts, belong to source_claim, and are not the result of third-party independent verification.

Why can a 35B model be compressed to run in 3G of memory? The explanation given in the report is that under the MoE architecture only about 3B parameters are activated per inference, plus layer-by-layer mixed-precision quantization compresses expert weights even smaller, and a contiguous weight storage layout reduces scattered reads. But the 3G memory figure does not specify the specific quantization precision, context length, or device memory footprint range, and whether a 12GB-memory phone can run it stably and smoothly remains unclear.

The report shows on-device demos in an offline state: on iPhone 17 Pro Max, the model gave the answer of a minimum temperature of minus 37.5 degrees Celsius and the date December 6, 1812 for a map of Napoleon's march on Russia; for the same stop sign photo and the same question, the unoptimized model ran at about 2 tokens per second, while after optimization it reached 8 to 12 tokens per second. On iPad Pro M4, running the same Napoleon question produced a consistent answer.

These demos are tests conducted by the vendor or media themselves, and the test conditions and reproducibility have not been made public. Whether the decoding speed of over 13 tokens per second can be maintained across different devices and different context lengths lacks independent testing.

The report also mentions that on an iPad Pro M4 with 16GB of memory, the Lookahead Expert Scheduling scheme improved decoding speed by about 5x overall; in tests on an M5 Mac with 16GB of memory, after integrating the DSpark draft model, decoding speed further improved by an average of 17%. These effect data are likewise self-tested by the team, with no external controlled experiments.

The report gives 35B-A3B's benchmark scores: MMBench 91.72, MMStar 80.67, MMMU 80.22, MathVista 88.00, Video-MME 76.40, LongVideoBench 68.90. On OmniDocBench v1.6 it scored 94.24, and the report claims the other four models were all around 80, a lead of more than 12 points. On pure text capability, Chinese knowledge CMMLU was 90.21, and tool calling τ2-Bench was 84.65, which the report claims were both the highest in the table.

After visual compression was set to 16x, MMBench dropped from 91.72 to 90.87, and OCRBench dropped from 91.20 to 90.40; based on this, the report claims the saved compute came with almost no loss in accuracy. But whether the conclusion that accuracy barely drops at 16x visual compression holds on broader tasks remains to be verified.

These benchmark scores do not specify details such as evaluation configuration, prompts, or comparison model versions, making the horizontal leadership conclusions hard to independently verify. Whether each benchmark score was obtained under a unified evaluation protocol, and whether it was fairly compared with models of the same size, cannot be confirmed.

Lookahead Expert Scheduling (LES) exploits the similarity of input features between two adjacent layers: while the current layer is computing, it sends the routing input features into the next layer's router to predict in advance and prefetch the next layer's expert weights. In the team's tests, it could prepare more than 80% of the required experts in advance, with no additional training needed. The prediction is only responsible for moving data ahead of time and arranging computation order; which experts are ultimately used for each token is still determined by the model's original routing mechanism.

The team also trained a matching DSpark draft model and integrated it into on-device inference frameworks such as llama.cpp, where the draft model first proposes multiple tokens and then hands them to the main model for batch verification. In tests on an M5 Mac with 16GB of memory, decoding speed further improved by an average of 17%.

To address the forgetting of general capabilities after fine-tuning, ModelBest introduced Self-Replay Fine-Tuning: before fine-tuning, the model is asked to answer questions covering its original capabilities, and high-quality answers are selected and mixed into the new task training data. The report claims this method follows the standard fine-tuning process, requires no changes to model structure or loss function, and the replay data only needs to be generated once and can be reused. But the actual cost and effect differences between Self-Replay Fine-Tuning and online reinforcement learning or On-Policy Distillation lack independent evaluation.

MiniCPM-V 4.7 supports deployment on iOS, Android, and HarmonyOS; on the day of release, adaptation was completed for chip platforms including Rockchip, Houmo, Intel, AMD, and Arm; and inference frameworks support SGLang, vLLM, llama.cpp, and Ollama. The actual usability and performance of completing adaptation for multiple chip platforms on the day of release were not detailed.

The report cites an IDC forecast that in 2026 China's new-generation AI phone shipments will reach 147 million units, up 31.6% year over year, with penetration rate exceeding 53% for the first time. ModelBest CEO Li Dahai said that 2026 is the first year of large-scale on-device AI deployment.

The report also mentions that memory prices continue to rise, 12GB has become the passing line for AI phones, flagships generally come standard with 16GB, and some models are starting to push toward 24GB. The iPhone 17 Pro Max has 12GB of memory, while 35B-A3B can run smoothly with about 3G of memory. In the longer term, the competitive focus of on-device AI is shifting from chip process nodes and compute power to whether models and inference frameworks can work well with hardware.