Run Xiaomi MiMo-V2.6 9B Locally: Ollama, llama.cpp and LM Studio Setup (8 GB GPU)

Xiaomi’s MiMo-V2.6 9B (full name MiMo-V2.6-Distill-Qwen-9B) landed in the official Ollama library yesterday, and it is the most interesting small model of the autumn. Xiaomi took Qwen3.5 9B and fine-tuned it on 77 billion tokens of data written by its much larger MiMo-V2.6 models. The result is a 9B model that can see images, call tools and think step by step, scores 44.6 on SWE Pro (up from 32.0 for the base model), and fits on an 8 GB GPU. It is MIT licensed. This guide shows how to run MiMo-V2.6 9B locally with Ollama, llama.cpp and LM Studio, which GGUF file to download for your VRAM, how to turn thinking on and off, and how to plug it into a coding agent.

What Is MiMo-V2.6 9B?

MiMo is Xiaomi’s family of large language models. On 22 September 2026 Xiaomi open-sourced the MiMo-V2.6 series: two big natively multimodal models, MiMo-V2.6-Pro and MiMo-V2.6-Flash, plus a small distilled model and a set of reinforcement-learning research resources. Xiaomi says Pro scored 46 on the Artificial Analysis Intelligence Index, ahead of Kimi K3 (44) and GLM-5.3 (45).

The small one is MiMo-V2.6-Distill-Qwen-9B. Its model card describes it like this:

• Base: Qwen3.5 9B (Qwen’s hybrid Gated DeltaNet architecture, which is why the GGUF architecture is listed as qwen35).
• Training: supervised fine-tuning on 77.4B tokens generated by MiMo models (27.2B loss-bearing). The mix was 23.2B tokens of code, 22.0B general agent work, 21.2B visual tasks and 11.0B cybersecurity.
• Abilities: coding, general agent tasks, “visual coding” (turning screenshots and mock-ups into code), cybersecurity, tool use and a thinking mode.
• Input: text and images. Context: 256K tokens in the Ollama build.
• Licence: MIT, so commercial use is fine.

Xiaomi calls it “a starting point for open research in agentic reinforcement learning”, but you do not need to care about the research angle. For a normal user it is simply a much better Qwen3.5 9B for code and agents.

Benchmarks vs Qwen3.5 9B

These are the numbers Xiaomi publishes on the model card. Rows marked † are Xiaomi’s internal test sets, so give them less weight than the public ones.

What this tells me: the gains are small on classic single-issue bug fixing (SWE Verified) but huge on longer, messier work: SWE Pro, terminal tasks, tool use and office automation. That is exactly what agent frameworks need, and it is where small models usually fall apart.

VRAM and Which GGUF to Download

bartowski has published imatrix GGUF quants of the model. These are the useful ones (sizes from the repo):

Leave 1-2 GB of headroom for the context and the vision projector. Because Qwen3.5 uses linear-attention (DeltaNet) layers for most of the network, the KV cache grows much more slowly than on an older 8B transformer, so long contexts are cheaper than you might expect. My estimate for Q4_K_M is about 7-8 GB of VRAM at a 32K context, which fits an 8 GB card if nothing else is using it.

bartowski notes that these files use per-tensor layouts computed for this model, and reports a lower KL divergence than the standard layout at the same size (0.87x for Q4_K_M). In plain terms: his Q4_K_M is a bit closer to the original than a normal Q4_K_M would be. If quant names are new to you, see my GGUF quantization guide.

Run MiMo-V2.6 9B With Ollama

This is the easiest route. Make sure Ollama is up to date (new architectures need a recent build; see how to install or update Ollama), then:

ollama pull mimo-v2.6:9b
ollama run mimo-v2.6:9b

The library page lists the tag with vision, tools and thinking, a 256K context and a download between 6.6 and 8.7 GB depending on the quant variant you pick (the page lists 8 builds in total). Check that it runs fully on your GPU:

ollama ps

You want 100% GPU in the PROCESSOR column. If you see a CPU split, lower the context:

/set parameter num_ctx 16384

or set OLLAMA_CONTEXT_LENGTH=16384 before starting the server. To make a permanent variant with your favourite settings, create a Modelfile:

FROM mimo-v2.6:9b
PARAMETER num_ctx 32768
PARAMETER temperature 0.6
SYSTEM You are a careful senior software engineer. Explain changes before making them.

ollama create mimo-coder -f Modelfile
ollama run mimo-coder

The temperature of 0.6 is my starting point, borrowed from common Qwen-family thinking-mode defaults; Xiaomi does not publish recommended sampling values on the card. All other Ollama commands are in my Ollama cheat sheet.

Run It With llama.cpp

llama.cpp gives you the most control and is usually a little faster. The bartowski quants were built with release b10964, so download that build or newer from the llama.cpp releases page (pick the CUDA 12 or 13 zip on Windows with an NVIDIA card).

Recent llama.cpp builds use the unified llama command. This one line downloads the Q4_K_M file plus the vision projector and starts a server with a web UI at http://localhost:8080:

llama serve -hf bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF:Q4_K_M -ngl 99 -c 32768 -fa on

If you already downloaded the files, point at them directly:

llama serve -m MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf ^
  --mmproj mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf ^
  -ngl 99 -c 32768 -fa on --jinja

(Use \ instead of ^ on Linux and macOS. On older builds the command is llama-server.) What the flags do:

• -ngl 99: put every layer on the GPU.
• -c 32768: 32K context. Raise it if you have spare VRAM.
• -fa on: flash attention, less memory and faster on RTX cards.
• --jinja: use the chat template embedded in the GGUF, which matters for tool calls.

For a quick terminal chat instead of a server, use llama cli with the same flags. The server exposes an OpenAI-compatible API at http://localhost:8080/v1, which any agent tool can use. Not sure which runner suits you? Compare them in Ollama vs LM Studio vs llama.cpp.

Run It in LM Studio

1. Update LM Studio and its llama.cpp runtime (Settings → Runtime). An old runtime will not know the qwen35 architecture.
2. Open the search tab and type MiMo-V2.6-Distill-Qwen-9B. Pick the bartowski GGUF and choose Q4_K_M, Q6_K or Q8_0.
3. Load it with GPU offload set to max, context 16K-32K and flash attention on.
4. For images, make sure the mmproj file is downloaded too; LM Studio shows a vision icon on the model when it is.

vLLM, SGLang and Transformers

If you have a Linux box or WSL and want maximum throughput, the model card gives these commands. SGLang is what Xiaomi uses in its quick start:

sglang serve --model-path XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B --reasoning-parser mimo --host 0.0.0.0 --port 30000

pip install vllm
vllm serve "XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B"

Both load the full BF16 weights (about 18 GB), so they need a 24 GB GPU, or two 16 GB cards with tensor parallel. In Python with Transformers, load it with AutoProcessor and AutoModelForMultimodalLM and device_map="auto". There is also a Docker Model Runner command: docker model run hf.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B.

Thinking Mode On and Off

MiMo-V2.6 9B thinks before it answers by default. That improves hard coding and agent tasks but adds a lot of tokens to simple questions. How to switch it:

• Ollama CLI: /set nothink and /set think, or start with ollama run mimo-v2.6:9b --think=false.
• Ollama API: add "think": false to the request body.
• vLLM / SGLang: pass "chat_template_kwargs": {"enable_thinking": false}. With the mimo reasoning parser, the thinking text comes back in a separate reasoning_content field.
• llama.cpp: use --reasoning-budget 0 to disable thinking, or keep it on and let the web UI fold the reasoning away.

My rule: thinking on for multi-file code changes, debugging and agent runs; thinking off for chat, summaries and autocomplete.

Using Image Input

About a quarter of the training mix was visual, and “visual coding” is one of the headline skills, so try it with screenshots:

ollama run mimo-v2.6:9b "Turn this mock-up into a responsive HTML and CSS page" ./mockup.png

In llama.cpp, drag the image into the web UI or send it as an image_url in the OpenAI-style API. Good uses: UI mock-up to code, reading error screenshots, explaining a chart, or describing a diagram. For heavy OCR or long videos, a larger vision model is still better; see my best local vision LLMs list.

Use It as a Coding Agent (Cline, OpenCode, Claude Code)

A 9B model that does well on Terminal Bench and Toolathlon is exactly what you want behind a local coding agent. Two ways to connect it:

• Through Ollama: in Cline, Continue or Roo Code choose the Ollama provider, base URL http://localhost:11434, model mimo-v2.6:9b. Raise the context to at least 32K, because agents send long system prompts.
• Through llama.cpp: choose the “OpenAI compatible” provider, base URL http://localhost:8080/v1, any API key, and start the server with --jinja so tool calls are formatted correctly.

Ollama can also launch supported agent tools directly with ollama launch, for example ollama launch claude --model mimo-v2.6:9b or the same with opencode. I walk through the full Cline setup in my Cline + local LLM guide. Keep tasks small and focused: a 9B model is great at “fix this test” or “add this endpoint”, and weaker at “refactor the whole app”.

What About MiMo-V2.6 Pro and Flash?

The RAM figures for Flash are my estimate from the parameter count (roughly 0.6 bytes per parameter at Q4 plus context). Xiaomi reports that both big models went through 30 RL steps in under six days using about 750,000 trajectories.

MiMo-V2.6 9B vs Other Small Models

For pure coding and tool use on an 8 GB card, MiMo is my pick. For chat in many languages, stay with Qwen3.5 9B or Gemma 4 12B. The full comparison by VRAM is in my best Ollama models guide.

Troubleshooting

• “unknown model architecture: qwen35”: your llama.cpp, LM Studio runtime or Ollama is too old. Update to llama.cpp b10964 or newer.
• Images are ignored: the mmproj file is missing. Add --mmproj in llama.cpp, or use -hf, which downloads it for you.
• Very long answers with lots of “thinking”: turn thinking off for simple tasks (see above).
• Tool calls come back as plain text: start llama.cpp with --jinja, and in Ollama make sure you use the official mimo-v2.6 tag, not a random GGUF import without a template.
• Slow on an 8 GB card: run ollama ps. If part of it is on the CPU, drop to Q4_K_M or IQ4_XS, or lower the context.

Frequently Asked Questions

What is MiMo-V2.6 9B?

It is Xiaomi’s MiMo-V2.6-Distill-Qwen-9B, a Qwen3.5 9B model fine-tuned on 77.4B tokens generated by the larger MiMo-V2.6 models. It handles text and images, supports tools and thinking, and is focused on coding and agent tasks.

How do I run MiMo-V2.6 in Ollama?

Update Ollama and run ollama run mimo-v2.6:9b. The download is 6.6 to 8.7 GB and the context window is 256K tokens.

How much VRAM does MiMo-V2.6 9B need?

About 7-8 GB at Q4_K_M with a 32K context, and about 11-12 GB at Q8_0. It runs on 8 GB GPUs at Q4 and on 6 GB GPUs at IQ4_XS or Q3_K_M.

Is MiMo-V2.6 9B free for commercial use?

Yes. The model is released under the MIT licence.

Can MiMo-V2.6 9B read images?

Yes. It accepts image input. In llama.cpp you need the mmproj file; in Ollama it works out of the box.

Can I run MiMo-V2.6 Pro or Flash locally?

Not on a normal PC. Pro has 1.02T parameters and Flash 309B. In Ollama both are cloud-only tags. Flash can run locally with llama.cpp only on machines with a few hundred GB of RAM.

Which llama.cpp version supports MiMo-V2.6 9B?

The bartowski GGUF quants were made with llama.cpp b10964, so use that release or newer.

Final Thoughts

MiMo-V2.6 9B is the best reason in months to give a small model another try. It is not smarter than a 27B model, but on agent-style work it closes much of the gap, and it does it in under 6 GB at Q4 with an MIT licence. Start with ollama run mimo-v2.6:9b, switch thinking off for chat, and give it real coding tasks through Cline or OpenCode.

Next steps: compare it with bigger options in the best Ollama models by VRAM, and when you are done testing models, free up disk space with how to delete Ollama models.

Sources

• XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B (Hugging Face)
• bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF
• Ollama library: mimo-v2.6
• TechNode: Xiaomi open-sources MiMo-V2.6 models (22 Sep 2026)
• Ollama library: mimo-v2.6-pro and mimo-v2.6-flash cloud tags
• llama.cpp