OpenBMB’s MiniCPM-V 4.7 is one of the best vision language models you can run on your own PC right now. It comes in two sizes: a 35B mixture-of-experts version with only about 3B parameters active per token, and a tiny 1B version that runs on almost anything. Both read images, multiple images, documents and video, both have a 256K token context, and both are Apache 2.0. Official GGUF builds work with Ollama and llama.cpp today. This guide explains which version and quant fits your GPU, the exact commands for Ollama, llama.cpp and vLLM, the settings OpenBMB recommends, and how to get it running well on a 16 GB card.
- MiniCPM-V 4.7 comes as 35B-A3B (MoE, first MoE in the MiniCPM-V line) and 1B (dense). Both are Apache 2.0 with a 262,144 token context.
- Fastest way to run it:
ollama run hf.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf:Q4_K_M, or the same tag withllama-server -hf. - The Q4_K_M GGUF is 21.2 GB, which fits a 24 GB card. On 16 GB, use IQ3_XS (14.5 GB) or IQ3_M (15.4 GB), or keep Q4_K_M and offload some experts to system RAM.
- The 1B model needs only about 5 GB of VRAM in llama.cpp and also runs on CPU.
- Recommended sampling: temperature 0.7, top_p 0.95, top_k 100, repetition penalty 1.03.
What Is MiniCPM-V 4.7?
MiniCPM-V is a family of compact vision language models from OpenBMB (a group linked to Tsinghua University and ModelBest) that has long punched above its size, especially on phones. Version 4.7 appeared on Hugging Face on October 6 and 7, 2026, and the model card with an Apache 2.0 license followed on October 12.
The big change is the backbone. The 35B version is built on a Qwen3.5-style sparse MoE text model with 256 experts, 8 of them used per token plus a shared expert. Most layers use linear attention, with a full attention layer every fourth layer, which keeps the KV cache small even with long contexts. A shared 27-layer vision tower handles images up to 980 pixels per slice, and it can compress visual tokens 16x by default or 4x when you need finer detail.
| MiniCPM-V 4.7 35B-A3B | MiniCPM-V 4.7 1B | |
|---|---|---|
| Type | Sparse MoE (256 experts, 8 active + shared) | Dense |
| Total parameters | 35.2B | About 1B |
| Active per token | About 3B | All |
| Layers | 40 (3 linear-attention : 1 full-attention) | 24 |
| Context | 262,144 tokens | 262,144 tokens |
| Inputs | Text, single image, multi-image, video | Text, single image, multi-image, video |
| BF16 weights | About 70 GB | 3.1 GB |
| License | Apache 2.0 | Apache 2.0 |
OpenBMB claims the 35B-A3B, in instruct mode, scores higher overall than Qwen3.6 35B-A3B while using less inference time, and that the 1B clearly beats Qwen3.5 0.8B. Those results are only published as charts, and independent leaderboards have not caught up yet, so treat them as vendor claims. OpenBMB also reports 13+ tokens per second for the 35B on an iPhone 17 Pro Max using about 3 GB of memory, thanks to expert scheduling, per-layer quantization and its DSpark speculative decoding.
35B-A3B or 1B: Which Should You Run?
- Run the 35B-A3B if you have a GPU with 16 GB or more, or a Mac with 32 GB or more. It is the one to use for document reading, charts, screenshots, detailed image questions and video.
- Run the 1B on laptops, older GPUs, CPUs or small devices, or for fast batch jobs like captioning thousands of images where speed matters more than depth.
Because only about 3B parameters are active per token, the 35B generates text at a speed closer to a small model than a 35B dense model. Memory, though, scales with the full 35B, which is why quant choice matters.
Which Quant Fits Your GPU?
OpenBMB publishes an official GGUF repo, openbmb/MiniCPM-V-4.7-35B-A3B-gguf. These are the file sizes it lists:
| Quant | File size | Good fit |
|---|---|---|
| IQ3_XS | 14.5 GB | 16 GB GPUs, short context |
| IQ3_M | 15.4 GB | 16 GB GPUs, tight |
| IQ4_XS | 18.7 GB | 20 to 24 GB GPUs |
| Q4_K_S | 19.9 GB | 24 GB GPUs |
| Q4_K_M | 21.2 GB | 24 GB GPUs (the default tag) |
| Q5_K_M | 24.7 GB | 32 GB GPUs, or 24 GB with offload |
| Q6_K | 28.5 GB | 32 GB GPUs |
| Q8_0 | 36.9 GB | 48 GB GPUs or big Macs |
| F16 | 69.4 GB | Workstations and servers |
Remember to leave room for the vision projector, the KV cache and your desktop. A rule of thumb: the file should be at least 1.5 to 2 GB smaller than your VRAM for comfortable use with images. On Apple Silicon, a 32 GB Mac is the practical floor for Q4_K_M, and there is an MLX 4-bit build (mlx-community/MiniCPM-V-4.7-35B-A3B-4bit) according to community listings.
There are also FP8, int4, AWQ and GPTQ repos for vLLM and SGLang users, and matching -1B repos for the small model.
Run MiniCPM-V 4.7 With Ollama
Ollama can pull GGUF files straight from Hugging Face, and this is the command from the official GGUF repo:
ollama run hf.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf:Q4_K_MSwap Q4_K_M for another quant tag from the table, for example IQ3_M on a 16 GB card. The model card also points to an Ollama library page at ollama.com/openbmb/minicpm-v4.7.
To ask about an image in the Ollama terminal, include the file path in your message:
>>> What does this chart show? ./sales-q3.pngOr call it from the API with a base64 image in the images field, which works with tools like Open WebUI out of the box. Update Ollama to the latest version first, since a new MoE plus linear-attention architecture needs recent runtime support.
Run It With llama.cpp
With a recent llama.cpp build, the -hf flag downloads the model for you:
# OpenAI-compatible server with a built-in web UI
llama-server -hf openbmb/MiniCPM-V-4.7-35B-A3B-gguf:Q4_K_M -c 16384 -ngl 99 --port 8080Then open http://localhost:8080 and drag an image into the chat. -c 16384 sets a 16K context (raise it if you have spare VRAM), and -ngl 99 puts all layers on the GPU.
If you build from source, update to the latest commit, because support for new architectures lands on master first:
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build -j --target llama-server llama-cliFor the 1B model, use the matching openbmb/MiniCPM-V-4.7-1B GGUF repo. One hands-on test reported just over 5 GB of VRAM for the 1B in llama.cpp, and it can run on CPU alone if needed.
Running the 35B on a 16 GB GPU
A 16 GB card like my RTX 5080 cannot hold the 21.2 GB Q4_K_M file entirely in VRAM. You have two good options:
Option 1: a smaller quant
IQ3_XS (14.5 GB) fits with a modest context. Quality drops a little compared with Q4, but everything stays on the GPU, so it is fast. This is the simplest choice for Ollama users.
Option 2: Q4_K_M with experts in system RAM
Because this is an MoE model, you can keep the shared layers and attention on the GPU and move some expert weights to system RAM. Only 8 of 256 experts are used per token, so the slowdown is smaller than with a dense model. In llama.cpp the --n-cpu-moe option controls how many layers’ experts stay on the CPU:
llama-server -hf openbmb/MiniCPM-V-4.7-35B-A3B-gguf:Q4_K_M
-c 16384 -ngl 99 --n-cpu-moe 12 --port 8080Start around 10 to 14 and lower the number until VRAM is nearly full. You need enough system RAM to hold the offloaded part, so 32 GB is the minimum and 64 GB is comfortable. This is the same expert offloading idea I explained for giant models in Can You Run Meta Muse Spark Open Locally?, just at a scale that suits a gaming PC.
vLLM, SGLang and Transformers
If you have a big GPU or want an API for many users, the model card lists these:
# vLLM
pip install vllm
vllm serve "openbmb/MiniCPM-V-4.7-35B-A3B"
# SGLang
pip install sglang
python3 -m sglang.launch_server --model-path "openbmb/MiniCPM-V-4.7-35B-A3B" --host 0.0.0.0 --port 30000
# Transformers serve
pip install "transformers[serving]>=5.18.0"
transformers serve openbmb/MiniCPM-V-4.7-35B-A3B --port 8000 --host 0.0.0.0The full BF16 weights are about 70 GB, so these commands need a data-center class GPU. On a single consumer card, point vLLM at the AWQ, GPTQ or int4 repos instead. Docker users can also run docker model run hf.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf:Q4_K_M.
Recommended Settings for Images and Video
OpenBMB’s recommended sampling for instruct mode:
| Setting | Value |
|---|---|
| temperature | 0.7 |
| top_p | 0.95 |
| top_k | 100 |
| min_p | 0.0 |
| presence_penalty | 0 |
| repetition_penalty | 1.03 |
And the vision settings, which matter most if you use the Transformers or vLLM path:
| Setting | Default | Images | Video |
|---|---|---|---|
downsample_mode |
16x | 4x for small text and fine detail | 16x |
max_slice_nums |
9 | 36 | 1 |
max_num_frames |
128 | – | 128 |
stack_frames |
1 | – | 3 or 5 |
use_image_id |
True | True | False |
The 4x mode keeps four times as many visual tokens, which helps with dense screenshots, receipts and charts but uses more memory and time. If you change downsample_mode, pass it to both apply_chat_template and generate. The model card examples set enable_thinking=False for instruct mode.
What It Is Good At
- Documents and OCR: reading PDFs, invoices, forms and screenshots, and turning them into text or JSON. OpenBMB ships task guides for OCR and PDF parsing.
- Charts and diagrams: explaining graphs and pulling numbers out of them.
- Multi-image comparison: spotting differences between product photos or UI versions.
- Video understanding: summarizing clips and answering questions about what happens, up to 128 frames.
- Visual grounding: locating objects in an image.
- Captioning datasets: the 1B is fast enough to caption thousands of images for LoRA training, for example for training a MOSS-Image LoRA with KohakuMoss.
One known weakness: reviewers note that, like earlier MiniCPM-V models, multilingual performance is weaker than English and Chinese.
MiniCPM-V 4.7 vs Other Local Vision Models
| Model | Size | Active | License | Best for |
|---|---|---|---|---|
| MiniCPM-V 4.7 35B-A3B | 35.2B MoE | About 3B | Apache 2.0 | Strong vision on a single consumer GPU |
| MiniCPM-V 4.7 1B | About 1B | All | Apache 2.0 | Laptops, CPUs, captioning at scale |
| MiniCPM-V 4.6 | 1.3B | All | Apache 2.0 | Previous small model, in Ollama since June 2026 |
OpenBMB’s own comparison target is Qwen3.6 35B-A3B, which it claims to beat in instruct mode with less inference time. I would wait for independent benchmarks before declaring a winner, but for anyone already running a 35B-A3B class model, MiniCPM-V 4.7 is worth a side-by-side test on your own images.
Troubleshooting
“Unknown model architecture” in llama.cpp or Ollama
Your runtime is too old. Update Ollama, or rebuild llama.cpp from the latest master.
The model answers but ignores images
The vision projector is not loaded. With -hf, llama.cpp downloads it automatically when the repo provides one. If you downloaded files by hand, pass the projector with --mmproj.
Out of memory with large images
Lower max_slice_nums, stay in 16x mode, reduce the context size, or pick a smaller quant.
Very slow on a 16 GB GPU
Ollama may be splitting the model between GPU and CPU without you noticing. Check with ollama ps. Switch to IQ3_XS to keep it all on the GPU, or use llama.cpp with --n-cpu-moe for more control.
Repetitive answers
Use the recommended sampling settings, especially repetition_penalty 1.03 and temperature 0.7.
FAQ
Is MiniCPM-V 4.7 free for commercial use?
Yes. Both the 35B-A3B and 1B model cards declare Apache 2.0.
How much VRAM does MiniCPM-V 4.7 need?
The 35B-A3B Q4_K_M file is 21.2 GB, so a 24 GB GPU is ideal. 16 GB works with IQ3 quants or expert offloading. The 1B needs only about 5 GB.
Is MiniCPM-V 4.7 in Ollama?
Yes, you can run the official GGUF directly with ollama run hf.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf:Q4_K_M, and the model card links an Ollama library page.
Can it understand video?
Yes, up to 128 frames by default. Use 1 slice per frame and stack 3 to 5 frames for best results.
Does it run on a Mac?
Yes. A 32 GB Mac is the practical minimum for Q4_K_M, and community MLX 4-bit builds exist. Smaller Macs can run IQ3 quants or the 1B.
How long a context can I use?
The models support 262,144 tokens. The linear-attention layers keep the KV cache small, but start at 16K to 32K on consumer GPUs and raise it if you have room.
Final Thoughts
MiniCPM-V 4.7 brings a genuinely capable, Apache-licensed vision model to ordinary PCs, and its MoE design keeps it fast. If you have a 24 GB card, run the Q4_K_M GGUF with one Ollama command. On a 16 GB card like mine, start with IQ3_XS for simplicity, or Q4_K_M with --n-cpu-moe in llama.cpp for better quality. And if you just need a fast captioner or a model for a laptop, the 1B is excellent value at around 5 GB of VRAM.
More local AI guides:
- Can You Run Meta Muse Spark Open Locally?
- Mistral Large 4 Open Weights: Release Date, Specs and Hardware
- Train a MOSS-Image LoRA With KohakuMoss, where MiniCPM-V can caption your dataset.
- Comfy Agent Guide