Run GLM-4.7-Flash Locally (2026): Ollama, llama.cpp, VRAM Table and Is GLM-5.3-Flash Worth the Upgrade?

GLM-4.7-Flash is still one of the most searched local models of 2026, and for good reason: Z.ai’s 30B mixture-of-experts only uses about 3B parameters per token, so it runs fast on a single gaming GPU while scoring 59.2% on SWE-bench Verified. In August Z.ai followed it with GLM-5.3-Flash, a far bigger 320B model that also has open weights. This guide shows how to run GLM-4.7-Flash locally with Ollama, llama.cpp and LM Studio on 12, 16 and 24 GB cards, with the GGUF sizes, the sampling settings that stop it looping, and the MoE offload trick that makes a 19 GB model usable on 16 GB. At the end I explain what you actually need to run GLM-5.3-Flash at home.

What GLM-4.7-Flash Is

Z.ai (formerly Zhipu AI) released GLM-4.7-Flash on 19 January 2026 as the small, local-friendly member of the GLM-4.7 family. The facts that matter:

• 30B-A3B mixture-of-experts: 30B parameters in total, about 3B active per token (5 of 64 experts).
• Multi-head latent attention (MLA), which keeps the KV cache small for long contexts.
• Context: 202,752 tokens maximum (Ollama lists 198K).
• Text in, text out, with tool calling and a thinking mode.
• Open weights on Hugging Face (zai-org/GLM-4.7-Flash). Z.ai has released its models under the MIT licence since July 2025; check the model card for the exact terms before commercial use.
• A cheap cloud option too: Z.ai lists a free API tier (one concurrent request) and paid pricing around $0.07 per million input and $0.40 per million output tokens.

The MoE design is the reason it is so popular locally. Memory is set by the 30B total weights, but speed is set by the 3B active ones, so it generates text far faster than a dense 27B-32B model of the same file size.

Benchmarks vs Qwen3 30B-A3B and GPT-OSS-20B

Z.ai’s numbers, as shown on the Ollama model page:

The pattern is clear: similar to its rivals on maths and pure reasoning, and far ahead on agentic work (fixing real GitHub issues, tool use, web browsing). That is why people use it as a local coding agent. It is no longer the newest small coder, though. Qwen3.8-27B and MiMo-V2.6 9B beat it on several coding benchmarks; see my best Ollama models ranking for how they compare.

VRAM Table: Which File Fits Your GPU

GGUF sizes from bartowski/zai-org_GLM-4.7-Flash-GGUF and Ollama’s tags. Unsloth’s own repo (unsloth/GLM-4.7-Flash-GGUF) has similar sizes with its Dynamic UD- quants; Unsloth recommends at least 4-bit and says the 4-bit build needs about 18 GB of combined RAM and VRAM.

“MoE offload” means keeping some expert layers in system RAM, explained below. Because only 3B parameters are active per token, this costs far less speed than offloading a dense model. For the best speed, Unsloth advises that VRAM plus RAM should be larger than the file, so nothing has to be read from disk. My GGUF quantization explainer covers the quant names.

Run GLM-4.7-Flash With Ollama

Ollama’s official tag needs Ollama 0.14.3 or newer. Update first (my Ollama install guide shows how), then:

ollama run glm-4.7-flash          # q4_K_M, 19 GB
ollama run glm-4.7-flash:q8_0     # 32 GB
ollama run glm-4.7-flash:bf16     # 60 GB

Ollama splits the model between GPU and CPU automatically when it does not fit; check with ollama ps. Raise the context for coding work with /set parameter num_ctx 32768 or the OLLAMA_CONTEXT_LENGTH environment variable. The tag supports tools and thinking, so it works with agents that talk to Ollama’s API.

One caution: Unsloth does not recommend loading its GGUF into Ollama, because of possible chat template problems. Use Ollama’s own glm-4.7-flash tag in Ollama, and use Unsloth or bartowski GGUFs in llama.cpp, LM Studio or Jan. All Ollama commands are in my Ollama cheat sheet.

Run It With llama.cpp (Recommended)

llama.cpp gives the best control over offload. Use a build from after 21-22 January 2026: early builds used the wrong expert scoring function (the fix sets it to sigmoid) and had slow flash attention on CUDA. If you downloaded GGUFs in the first days after release, download them again, because the files were re-made with the fix.

Quick start, letting llama.cpp download the file:

llama-cli -hf unsloth/GLM-4.7-Flash-GGUF:UD-Q4_K_XL --ctx-size 16384 --temp 1.0 --top-p 0.95 --min-p 0.01

OpenAI-compatible server for a 24 GB card, everything on the GPU:

llama-server -hf unsloth/GLM-4.7-Flash-GGUF:UD-Q4_K_XL -ngl 99 --ctx-size 32768 --temp 1.0 --top-p 0.95 --min-p 0.01 --repeat-penalty 1.0 --jinja --port 8001

--jinja uses the chat template stored in the GGUF, which tool calling needs. Note that llama.cpp’s default min_p is 0.05, so set 0.01 explicitly. Recent llama.cpp versions also accept the unified llama serve form.

The MoE Offload Trick for 12-16 GB Cards

In a mixture-of-experts model, most of the file is expert weights, and each token only uses a few of them. llama.cpp can keep the attention layers and shared parts on the GPU and move the expert tensors of some layers to system RAM. The flag is --n-cpu-moe N, where N is the number of layers whose experts stay on the CPU:

# 16 GB GPU with Q4_K_M (18.47 GB): keep a few layers' experts in RAM
llama-server -m zai-org_GLM-4.7-Flash-Q4_K_M.gguf -ngl 99 --n-cpu-moe 8 --ctx-size 32768 --temp 1.0 --top-p 0.95 --min-p 0.01 --jinja

# 12 GB GPU: move more experts to RAM
llama-server -m zai-org_GLM-4.7-Flash-Q4_K_M.gguf -ngl 99 --n-cpu-moe 20 --ctx-size 16384 --temp 1.0 --top-p 0.95 --min-p 0.01 --jinja

Start with a high number, then lower it step by step until you run out of VRAM; the lowest number that loads is the fastest. You need enough system RAM for the part you offload (32 GB of RAM is comfortable for Q4). This works for any MoE GGUF, including the much bigger models in my Kimi K3 hardware guide.

LM Studio and Mac

• LM Studio: search “GLM-4.7-Flash” in Discover, download a Q4 GGUF, and in the load settings set GPU offload to max. Newer LM Studio versions have an option to force MoE experts onto the CPU, which does the same job as --n-cpu-moe. Set the sampling values from the next section in the model’s preset.
• Mac: a Mac with 32 GB of unified memory runs the 4-bit build well, and Apple Silicon is where the published speed numbers are best (Techloy reports about 82 tokens per second on an M4 Max MacBook Pro). MLX versions are also available in LM Studio.

Not sure which runner suits you? See Ollama vs LM Studio vs llama.cpp.

Best Settings and How to Stop Looping

These are Unsloth’s recommendations for GLM-4.7-Flash. The most common complaint, endless loops or repeated paragraphs, usually comes from one of three things:

1. An old llama.cpp build or old GGUF from before the 21 January scoring-function fix. Update both.
2. A repeat penalty above 1.0. Many apps default to 1.1; Unsloth says to disable it because it makes repetition worse with this model.
3. Too small a context. Thinking output is long. Below 16K tokens the model can lose the start of its own reasoning.

Thinking is on by default in the chat template (the prompt ends with a <think> tag). For quick answers, turn it off in your app if it offers a switch, or ask for a short answer in the system prompt.

Use It as a Local Coding Agent

GLM-4.7-Flash’s strong SWE-bench and tau2 scores make it a good free back end for coding agents. Start llama-server (or Ollama), then point your tool at the local OpenAI-compatible endpoint:

• OpenCode, Cline, Continue, Roo Code: add a custom OpenAI-compatible provider with base URL http://localhost:8001/v1 (llama.cpp) or http://localhost:11434/v1 (Ollama) and model name glm-4.7-flash.
• Use the tool-calling settings (temp 0.7, top_p 1.0) and at least a 32K context, because agents send large prompts with file contents and tool results.
• Keep --jinja on in llama.cpp so tool calls are formatted correctly.

I use the same pattern with a bigger model in my Blender MCP with a local LLM in Cline guide; the configuration steps are identical.

GLM-5.3-Flash: Can You Run It at Home?

Z.ai put GLM-5.3-Flash on Hugging Face on 25 August 2026 under the MIT licence. Despite the “Flash” name it is a completely different size class:

Z.ai reports that GLM-5.3-Flash beats GLM-5.2 on all six benchmarks it published (for example DeepSWE 63.4 vs 46.2) and edges Claude Opus 4.8 on DeepSWE (63.4 vs 58.0), while trailing slightly on Terminal Bench (84.3 vs 85.0). Those are the vendor’s own runs.

To run it at home you need a lot of system RAM: Unsloth sets a floor of about 100 GB of RAM for the 1-bit quant and 128 GB for 3-bit, Atomic Chat recommends 256 GB for 4-bit, and nothing below 64 GB works. With 128 GB of RAM and one or two 16-24 GB GPUs, a 1-2 bit build with all experts in RAM is possible but slow, and you need Unsloth’s llama.cpp fork until support lands in mainline. For most people, GLM-4.7-Flash locally plus GLM-5.3-Flash through the cheap API is the practical combination. If you are planning a big-RAM build, the same rules apply as in my Kimi K3 VRAM and RAM table.

Troubleshooting Common Problems

Ollama says the model is not supported or the pull fails

GLM-4.7-Flash needs Ollama 0.14.3 or newer. Check with ollama --version, update from ollama.com/download (or rerun the install script on Linux), then pull again. My Ollama install guide covers updating on every platform.

llama.cpp: “unknown model architecture” or the GGUF will not load

Your llama.cpp build is older than the model. Download the latest release from the llama.cpp releases page, and check you downloaded the complete file (compare the size with the Hugging Face listing; a cut-off download fails the same way).

CUDA out of memory on a 12-16 GB card

Move more experts to system RAM with a higher --n-cpu-moe value (see the MoE offload trick), lower the context with -c 16384 or -c 32768, use one slot with -np 1, and close other GPU apps such as ComfyUI or games. A smaller quant (Q3_K_M instead of Q4_K_M) is the last step, not the first, because offloading experts costs less quality than shrinking them.

It repeats itself or never stops

Use the settings from Best Settings: temperature 1.0, top_p 0.95, min_p 0.01, and keep the repeat penalty at 1.0. Raising the repeat penalty, a common habit from older models, makes GLM-4.7-Flash worse. Also make sure your app uses the model’s own chat template (--jinja in llama.cpp); a wrong template is the other main cause of endless output.

Tool calls come back broken in agents

Switch to the tool-calling settings (temperature 0.7, top_p 1.0), start llama.cpp with --jinja, and update llama.cpp or Ollama, since tool-call parsing for new models often improves in later builds. Test one simple tool before connecting a full agent.

Much slower in LM Studio than in llama.cpp

GUI apps do not always offload MoE experts the smart way. In LM Studio’s model load settings, turn on the option to keep MoE expert weights on the CPU and set GPU offload to the maximum, or run llama.cpp directly with --n-cpu-moe for full control. My Ollama vs LM Studio vs llama.cpp comparison explains the trade-offs.

Frequently Asked Questions

Can I run GLM-4.7-Flash locally?

Yes. It is an open-weight 30B-A3B MoE. Run it with ollama run glm-4.7-flash (19 GB, Ollama 0.14.3 or newer), or with a GGUF in llama.cpp, LM Studio or Jan.

How much VRAM does GLM-4.7-Flash need?

About 18-19 GB for 4-bit, so a 24 GB GPU runs it fully. On 16 GB use Q3_K_M (14.07 GB) or Q4 with some experts offloaded to RAM using –n-cpu-moe. On 12 GB use Q2_K or heavier expert offload.

Does GLM-4.7-Flash run on a 16 GB GPU?

Yes. Because only 3B parameters are active per token, keeping a few expert layers in system RAM costs little speed. Q3_K_M also fits fully on the GPU with an 8K-16K context.

What are the best settings for GLM-4.7-Flash?

Temperature 1.0, top_p 0.95 and min_p 0.01 for general use; temperature 0.7 and top_p 1.0 for tool calling. Turn the repeat penalty off (1.0).

Why does GLM-4.7-Flash keep repeating itself?

Usually an old llama.cpp build or GGUF from before the January 21 fix, a repeat penalty above 1.0, or a context that is too small for its thinking. Update, set repeat penalty to 1.0 and use at least 16K context.

Is GLM-5.3-Flash open source and can I run it locally?

Its weights are on Hugging Face under the MIT licence, but it has 320B parameters. Even the 1-bit GGUF is 93.1 GB, so you need roughly 100-128 GB or more of RAM, and mainline llama.cpp support was still pending at the time of writing.

Is GLM-4.7-Flash good for coding?

Yes. Z.ai reports 59.2% on SWE-bench Verified and 79.5 on tau2-Bench, far ahead of Qwen3-30B-A3B-Thinking and GPT-OSS-20B on agentic coding. Newer models such as Qwen3.8-27B now score higher.

Final Thoughts

GLM-4.7-Flash is still a great local model: fast because only 3B parameters are active, very good at agentic coding, and small enough for a 24 GB card at 4-bit or a 16 GB card with MoE offload. Update llama.cpp, set repeat penalty to 1.0 and min_p to 0.01, and it behaves well. GLM-5.3-Flash is the more powerful model, but it is a 128 GB-RAM project, not a gaming PC one. For other models that fit 16 GB, see Qwen3.8-27B on a 16 GB GPU and Gemma 4 locally.

Sources

• Ollama library: glm-4.7-flash (tags, benchmarks, minimum version)
• Unsloth docs: GLM-4.7-Flash (settings, fixes, llama.cpp commands)
• bartowski/zai-org_GLM-4.7-Flash-GGUF (file sizes)
• Techloy: Zhipu AI launches GLM-4.7-Flash
• Atomic Chat: How to run GLM-5.3-Flash locally
• Wikipedia: Z.ai (GLM release history)