How to Run DeepSeek V4.1 Flash Locally (2026): GGUF Sizes, RAM Needed, llama.cpp and Engram SSD Offload

DeepSeek V4.1 Flash is one of the strongest open models you can download right now, and the October update, V4.1-Flash-1017, pushed its coding and agent scores up again. It is also huge: a 552B backbone plus 196B of “Engram” memory tables, and a full download of around 500 GB.

The good news is that Engram changes the maths. Those memory tables can stay on your SSD instead of in RAM, so a 1-bit build fits in about 128 to 145 GB of RAM plus VRAM.

This guide shows which quant fits which machine, the exact llama.cpp and Unsloth commands, the right sampling and thinking settings, and the problems people hit.

What V4.1 Flash 1017 Is

DeepSeek released V4.1 Flash on 10 September 2026, and the 1017 checkpoint followed on 17 October as a retrained “minor update”. The official model card calls it an update to V4.1-Flash, and Unsloth’s guide calls it the strongest open model so far for coding and agent work.

Only 8B to 16B parameters do work for each token, which is why it can run at a usable speed once the weights are in memory. The hard part is fitting the weights at all.

For a wider look at DeepSeek’s line-up and which smaller models fit normal PCs, see our run DeepSeek locally guide.

Why Engram Makes It Runnable

Engram is a large lookup table of n-gram embeddings. For each token, the model reads only a few rows of it. That access pattern means the table does not need to sit in RAM or VRAM.

With llama.cpp’s memory-mapped loading, the table stays on your SSD and rows load on demand into the operating system’s file cache. In the Unsloth GGUFs, the Engram table takes:

• 110.6 GB as IQ4_NL (in the 1-bit quants)
• 135.2 GB as Q5_K (2-bit quants)
• 161.3 GB as Q6_K (IQ3 quants)
• 202.8 GB as MXFP8 (Q3_K_XL and up)

So for memory planning, you subtract the Engram size from the download size. You still need the full download on disk, and a fast NVMe SSD matters, because the table is read from it constantly.

Quants, File Sizes and Memory

These are the Unsloth Dynamic quants. The last column is the part that must fit in RAM plus VRAM.

Unsloth adds that you need 5 to 10 GB above these numbers, plus more for long context. The KV cache itself is tiny, at roughly 890 bytes per token, so even a 1M context is under 1 GB.

New to quant names? Our GGUF quantization guide explains what IQ1, Q2_K and the rest mean.

Which Machines Can Run It

The only measured speeds I found come from community builds on two linked DGX Sparks: about 55 to 75 tokens per second for prose and 90 to 110 for code with speculative decoding, and 1,700 to 2,200 tokens per second for prompt processing.

A single Spark at 1.6-bit managed about 30 tokens per second on the September weights. These numbers come from the build authors and have not been independently repeated.

I have not run it myself, so I am not quoting a speed for a normal PC. With the backbone split between DDR5 and the GPU, expect it to be much slower than those Spark figures.

If you are planning a build, read our RAM for local AI guide and DGX Spark for local AI review.

Run It With llama.cpp

Mainline llama.cpp does not support V4.1 Flash yet. Support lives in PR #30817 (“model : add DeepSeek-V4.1-Flash”), opened by Unsloth’s Daniel Han on 19 October. When I checked, it was still a draft and unmerged, and its only backend label was CUDA. So you build from that branch.

1. Build the PR branch

On Linux or WSL with an NVIDIA GPU:

apt-get update
apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev git -y
git clone https://github.com/ggml-org/llama.cpp
git -C llama.cpp fetch origin pull/30817/head:deepseek41
git -C llama.cpp checkout deepseek41
cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split
cp llama.cpp/build/bin/llama-* llama.cpp

For CPU only, use -DGGML_CUDA=OFF. On Apple Silicon, Metal is on by default when CUDA is off.

2. Download one quant

pip install -U "huggingface_hub[cli]"
hf download unsloth/DeepSeek-V4.1-Flash-1017-GGUF --local-dir unsloth/DeepSeek-V4.1-Flash-1017-GGUF --include "*UD-Q2_K_XL*"

Swap UD-Q2_K_XL for the quant that fits your machine (on a 128 GB machine, UD-IQ1_S), and change the folder and file names in the run command below to match. Each quant is split into several files (UD-Q2_K_XL has seven). Keep all the parts in one folder.

3. Run it

./llama.cpp/llama-cli \
  --model unsloth/DeepSeek-V4.1-Flash-1017-GGUF/UD-Q2_K_XL/DeepSeek-V4.1-Flash-1017-UD-Q2_K_XL-00001-of-00007.gguf \
  --temp 1.0 --top-p 1.0 --top-k 0 --min-p 0.0 \
  --chat-template-kwargs '{"reasoning_effort":75}' \
  --load-mode mmap --lazy-mode on \
  --override-tensor "per_layer_token_embd.weight=CPU"

Point --model at the first shard, ending in 00001-of-0000N.gguf. Keep --load-mode mmap and --lazy-mode on: these are what let the Engram table stay on the SSD.

For a local OpenAI-compatible API (for coding tools like Cline or Continue), use llama-server with the same flags.

The PR also adds --mmproj for the vision projector and --dspark for DeepSeek’s speculative drafter. The drafter is a separate download (about 8 GB at Q8_0) and Unsloth says it gives about 1.3 to 1.5x faster decoding.

Run It With Unsloth Desktop

Unsloth recommends its own Unsloth Desktop app (macOS, Windows and Linux) as the easiest route, and claims up to about 3x faster inference than other CPU and VRAM offload setups. The serve command is:

unsloth run --model unsloth/DeepSeek-V4.1-Flash-1017-GGUF:UD-Q2_K_XL \
  --temp 1.0 --top-p 1.0 --min-p 0.0 \
  --chat-template-kwargs '{"reasoning_effort":75}'

This is the simplest option on Windows, where building a llama.cpp PR branch by hand is more work.

Ollama, LM Studio and Mac Options

• Ollama: I could not find an official Ollama library entry for V4.1 Flash. Ollama would need the new llama.cpp code first, so use llama.cpp or Unsloth Desktop for now.
• LM Studio: its V4.1 Flash page offers hosted LM Studio Cloud inference with 1M context and a 256K output budget, and reasoning levels Off, Low, Medium, High and Max. It does not list local quants yet.
• Mac: besides the GGUFs, there are MLX builds of the 1017 weights (about 313 GB at 2-bit and 432 GB at 4-bit), which need more than 256 GB, so in practice a 512 GB Mac. Our MLX vs GGUF guide explains which format is faster.
• Servers: DeepSeek’s card shows vllm serve and SGLang for multi-GPU servers. One guide cites a 614 GB VRAM minimum in vLLM’s recipe.

For how the three main local apps compare in general, see Ollama vs LM Studio vs llama.cpp.

Sampling and Thinking Settings

Thinking is on by default. The reasoning_effort value is a 1 to 100 scale for how thorough the reasoning is, not a token limit. The named levels are low = 50, medium = 62, high = 75 and max = 100.

# default thinking
--chat-template-kwargs '{"reasoning_effort":75}'
# maximum thinking
--chat-template-kwargs '{"reasoning_effort":100}'
# thinking off
--chat-template-kwargs '{"enable_thinking":false}'

On a slow local setup, turning thinking off or dropping to low is the biggest speed win, because the model writes far fewer tokens. Our reasoning effort guide covers how these levels change cost and quality.

One note for developers: the official repo has no Jinja chat template. It ships a Python reference encoder in its encoding folder. The GGUF builds include a template, which is why the --chat-template-kwargs flags above work.

Does 1-Bit or 2-Bit Hurt Quality?

Less than you would expect, according to Unsloth’s own tests. Treat these as vendor numbers.

The 2-bit Q2_K_XL is almost level with 8-bit on Unsloth’s average, so it is the sweet spot if you have about 200 GB of RAM plus VRAM. The 1-bit builds lose more, especially on coding (Aider pass@1 drops from 44.38 at IQ2_XXS to 36.52 at IQ1_S).

Low-bit quants also think for longer. In Unsloth’s tests, answers cut off at 32K tokens 1.48% of the time at Q8 and 4.15% at IQ1_S, so give small quants a bigger output budget.

You will also see third-party “GSQ-RCO” quants that claim to fit much smaller GPUs. Unsloth says it could not reproduce their accuracy claims, so I would stick with the Unsloth or official-format builds for now.

Benchmarks

DeepSeek’s own figures, comparing 1017 with the September V4.1 Flash:

The same card shows Claude Opus 5 ahead on several tests, including HLE (56.3) and Terminal-Bench 4.0 (51.8). For how DeepSeek stacks up against closed models in daily use, see DeepSeek vs Claude.

Troubleshooting

“unknown model architecture” in llama.cpp

You are on mainline llama.cpp, LM Studio’s bundled runtime, or an older build. Build the deepseek41 branch from PR #30817 as shown above, or use Unsloth Desktop.

It only loads part of the model or says a file is missing

All shards of a quant must be in the same folder, and --model must point at the first one. Check the download finished for every part.

It is extremely slow, or the SSD light never stops

That is the Engram table being read from disk. Use an NVMe SSD, not a SATA or USB drive. Make sure --load-mode mmap and --lazy-mode on are set, and close other apps so the OS has spare RAM to cache rows.

Out of memory at load time

The non-Engram part does not fit. Drop to a smaller quant (for example UD-IQ1_M to UD-IQ1_S), keep --override-tensor "per_layer_token_embd.weight=CPU", and shorten the context. Our VRAM usage guide helps you see what is filling the GPU.

Answers stop in the middle of thinking

The output budget is too small. Raise the max tokens (DeepSeek suggests 256K), or lower reasoning_effort.

Odd repetition or rambling

Check the sampling settings. Use temperature 1.0 and top_p 1.0 with top-k and min-p off. Lower temperatures than DeepSeek recommends can make long reasoning loop.

Not enough disk space

Even with SSD offloading, the whole quant has to be on disk: 233 GB for the smallest. Free up space or use a second NVMe drive for models.

FAQ

Can I run DeepSeek V4.1 Flash on a single RTX 4090 or 5090?

Not on the GPU alone. You need about 122 GB or more of combined RAM and VRAM even for the 1-bit build. A 32 GB card plus 128 GB of system RAM fits the smallest quant on paper.

Is DeepSeek V4.1 Flash free for commercial use?

Yes. The code and weights are under the MIT licence.

Which quant should I download?

The largest one whose “Rest” column fits in your RAM plus VRAM with 10 GB or more to spare. UD-Q2_K_XL is the best balance if you have about 210 GB. On a 128 GB machine, UD-IQ1_S is the only real option.

Does it support images?

Yes, it takes text and images. In llama.cpp, vision needs the --mmproj projector file and the llama-mtmd-cli tool from the PR build.

What is the difference between V4.1 Flash and V4.1-Flash-1017?

1017 is the 17 October retrain of the September model. It scores higher on reasoning, terminal and coding tests, with the same size and hardware needs.

Is there a cheaper way to use it?

The DeepSeek API is far cheaper than buying 200 GB+ of memory. See our DeepSeek API pricing guide and DeepSeek API Python tutorial.

What smaller local models are close to it?

For 16 to 24 GB GPUs, Qwen models are the usual pick. See Qwen3.8-27B on a 16 GB GPU and our best Ollama models by VRAM.

Sources

• DeepSeek-V4.1-Flash-1017 model card
• Unsloth DeepSeek V4.1 Flash guide and GGUF repo
• llama.cpp PR #30817
• LM Studio model page
• ModemGuides hardware reality check (community speed figures)