llama-modchip: Run DeepSeek V4-Flash 284B on 64 GB RAM With No GPU (Setup, Speed, Limits)

A new open-source project called llama-modchip claims something that sounded impossible a month ago: running the full 284B DeepSeek V4-Flash on a normal 64 GB PC, with no GPU at all.

The developer reports about 9 to 15 tokens per second on a 16-core Ryzen laptop chip. The trick is an aggressive 1.71-bit build plus patched llama.cpp CPU kernels.

This guide explains what llama-modchip is, what hardware you need, the exact commands to run it on Linux, Windows and Mac, how the quant works, the quality trade-offs, and which smaller models make more sense if you have 16 to 32 GB of RAM.

What llama-modchip Is

llama-modchip is a CPU-first fork of the llama.cpp toolchain. It adds new ggml kernels, a new quant type, a tiny speculative “drafter” model and a command-line runner, all aimed at sparse Mixture-of-Experts (MoE) models.

The repository appeared around 29 October 2026 and moved fast. Four releases landed between 2 and 4 November, and the current version is v2.4.0.

Release timeline

News coverage started on 8 and 9 November after the project was shared on Hacker News and X.

Why a 284B Model on 64 GB Matters

DeepSeek V4-Flash-0731 is the general-availability build of V4 Flash, released on 31 July 2026 under the MIT licence. It has a 1M-token context and is still one of the strongest open models for coding and agents.

Until now, the smallest common build was Unsloth’s UD-IQ1_S at 76.87 GiB. That is too big for a 64 GB PC, so the model streamed from SSD and crawled. llama-modchip cuts about 20 GiB, which is the difference between “does not fit” and “fits with a hair to spare”.

Note: this is the older 284B V4-Flash, not the newer V4.1 Flash. V4.1 Flash is a 552B model with Engram memory tables and needs 128 GB or more. We cover it in our DeepSeek V4.1 Flash local guide.

Supported Models and File Sizes

The project ships pre-quantized GGUFs on Hugging Face under the Synchopate account. These are the builds listed in the README.

The DeepSeek repo is Synchopate/DeepSeek-V4-Flash-0731-Fractal-64GB-CPU-GGUF. It holds three shards (DeepSeek-V4-Flash-0731-Fractal-00001-of-00003.gguf to -00003-of-00003.gguf) totalling 60,729,886,080 bytes, plus Linux binaries.

Speed Numbers by Hardware

Every number below is reported by the developer or by community members on the project’s own leaderboard. No outside outlet has re-run them yet.

So the 284B build is roughly 1.5 to 1.8 times faster than stock llama.cpp on the same box. About 10 tok/s is readable chat speed, but long agent runs will feel slow.

Tip: a “Turbo” profile with speculation is reported at up to 16 tok/s. Speed-ups that use fewer experts trade a little quality (see the quality section).

Will It Run on Your PC?

Three things decide it: total RAM, CPU instruction support and memory bandwidth. Here is a quick guide based on the README’s own tiers.

Warning: on a 64 GB PC, Windows Forum’s coverage reported only about 59.6 GiB addressable and 0.7-2 GiB left after loading. A browser with a few tabs can push the model into swap and drop speed to a crawl.

CPU requirements

• Best: CPUs with AVX-512BW, such as AMD Zen 4 and Zen 5 (Ryzen 7000/9000, 9955HX) and recent Intel workstation chips. The fastest kernels are written for it.
• Also covered: an AVX2 path for the Q2_0 kernels arrived in v2.3.0, so older chips can run, just slower.
• ARM: Apple M4 (NEON/AMX) and AWS Graviton3 are on the benchmark list.
• Cores: examples use 16 threads (-t 16). Match it to your physical cores.

Not sure how much RAM you need for local AI in general? Our RAM guide for local AI explains 32 vs 64 vs 128 GB with real numbers.

How to Run DeepSeek V4-Flash 284B on Linux

Linux is the main target, and the pre-built binaries are Linux x86_64 only. You need about 60 GB of free disk for the model files.

1. Check your CPU first. The project ships a readiness script that downloads nothing big: curl -sL https://raw.githubusercontent.com/synchopate/llama-modchip/main/scripts/check_cpu_readiness.py | python3. Read the script before piping it if you prefer.
2. Free your RAM. Close browsers and desktop apps. On a 64 GB box, a minimal desktop or a headless server works best.
3. Install the Hugging Face CLI. Run pip install -U huggingface_hub so the hf command exists.
4. Download the model and binary. Run hf download Synchopate/DeepSeek-V4-Flash-0731-Fractal-64GB-CPU-GGUF --local-dir ./dsv4.
5. Make the server executable. Run chmod +x ./dsv4/llama-server-modchip-linux-x86_64.
6. Start the server. Use the command below, then open http://localhost:8080 for the web chat or point any OpenAI-compatible app at http://localhost:8080/v1.

./dsv4/llama-server-modchip-linux-x86_64 \
  -m ./dsv4/DeepSeek-V4-Flash-0731-Fractal-00001-of-00003.gguf \
  -c 32768 -t 16 -tb 16 -b 2048 -ub 512 -fa on --port 8080 --jinja

The example uses a 32K context (-c 32768). The 256K figure in the README is the tested maximum, so raise it only once the 32K run is stable.

The easy route: the modchip script

If you would rather not handle files yourself, clone the repo and let its launcher do the work:

git clone https://github.com/synchopate/llama-modchip.git
cd llama-modchip
./modchip chat

Other useful commands are ./modchip web (browser UI), ./modchip bench (speed test), ./modchip status and ./modchip stop.

Start smaller: the 30B model

For a first test on 16-32 GB, the 30B Nemotron download is a quarter of the size:

hf download Synchopate/NVIDIA-Nemotron-3.5-30B-A3B-Fractal-64-CPU-GGUF \
  llama-server-modchip-linux-x86_64 NVIDIA-Nemotron-3.5-30B-A3B-Fractal-64.gguf --local-dir .
chmod +x llama-server-modchip-linux-x86_64
./llama-server-modchip-linux-x86_64 -m NVIDIA-Nemotron-3.5-30B-A3B-Fractal-64.gguf -c 262144 -t 16 --port 8080

Windows and Mac Options

• Windows 10/11: run windows/AVVIA_MODCHIP.bat from the cloned repo, or ./modchip start inside Git Bash. The v2.4.0 Windows pack is described as a hybrid GPU+CPU pack, and nobody has published 284B results on Windows yet.
• WSL2: a reasonable fallback for the Linux binaries, but WSL caps its RAM share by default. Raise memory= in %UserProfile%\.wslconfig first, and expect less headroom than native Linux.
• macOS (Apple Silicon): the repo has an Apple Silicon folder with Metal and NEON/AMX paths. M4 Max and M4 Pro are on the benchmark list for 30B models.

Note: for the 284B model, Windows is the hardest case. The OS itself uses several GB, so a 64 GB Windows PC is very unlikely to have enough free memory. Use Linux or a machine with 96 GB or more.

Using It With Claude Code and Apps

The server speaks the OpenAI-compatible API, so most local-AI front ends can connect to http://localhost:8080/v1. Typical options:

• Open WebUI, Continue, Cline: add an OpenAI-compatible provider with base URL http://localhost:8080/v1 and any API key string.
• Claude Code: the project has a ./modchip claude command. It keeps KV-cache snapshots in /dev/shm so long Claude Code system prompts restore in about 13-33 ms instead of being re-read. For other routers, see our Claude Code Router guide.
• Your own scripts: use the standard OpenAI Python client with base_url="http://localhost:8080/v1".

Tip: at 10 tok/s, a 2,000-token answer takes over three minutes. For coding agents, use the 284B model for planning and hard bugs, and a fast 30B model for quick edits.

How the 1.71-Bit Quant Works

The project starts from Unsloth’s UD-IQ1_S GGUF and squeezes it further. It does not re-train anything. The model card describes this mix:

Fractal-64 packs weights in blocks of 64 with one BF16 scale per block. The author says the scales are chosen with “analytical SwiGLU-H2 second-order scale matching”, which aims to keep each expert’s output close to the original.

Speed comes from new CPU kernels. Coverage of v2.4.0 says they use four levels of tiling across AVX-512BW registers and the L1, L2 and L3 caches. A 1.64 MB n-gram drafter (“Stockfish-PVS”) then guesses upcoming tokens for speculative decoding.

New to quant names like IQ1_S and Q4_K? Our GGUF quantization guide explains them in plain words.

Does 1.71 Bits Hurt Quality?

This is the big open question. The README reports a logit cosine similarity of 1.00000 and zero KL divergence, and calls the default mode “lossless”.

Read that carefully. As CuriousLM pointed out, the comparison is between the patched and unpatched runtimes on the same 1.71-bit file. “Lossless” means the new kernels give the same output as stock llama.cpp, not that the file matches DeepSeek’s full-precision model.

• Known: any sub-2-bit model loses some accuracy versus FP8 or BF16, especially on long reasoning and tool use.
• Not yet published: task benchmarks (coding, maths, agents) for the 1.71-bit build against the full model.
• Self-reported: the 100/100 “quality gate” and the community leaderboard come from the project itself.

Warning: test it on your own prompts before relying on it for code or facts. If answers look off, compare a few against the DeepSeek API, which runs the full model.

llama-modchip vs Stock llama.cpp, Ollama and the API

If you just want a good local model with zero fuss, a 27-30B model in Ollama is still the easier path. Our best Ollama models list and Ollama vs LM Studio vs llama.cpp comparison help you pick.

Troubleshooting

“unknown tensor type 40”

You loaded a Fractal-64 file in stock llama.cpp, Ollama or LM Studio. Use the bundled llama-server-modchip binary or ./modchip instead.

Speed drops to 1-2 tok/s after a while

The system is swapping. Check with free -h and vmstat 1. Close apps, lower -c to 16384, or move to a machine with more RAM.

Process killed while loading

The Linux OOM killer stepped in. Run dmesg | tail to confirm, then free memory or add a small swap file as a safety net (it will be slow if used).

Much slower than the published numbers

Check that your CPU has AVX-512BW (grep -o avx512bw /proc/cpuinfo | head -1) and that -t matches your physical cores. Single-channel RAM also halves bandwidth, so use two or more sticks.

Speculative decoding gives odd results

The README says Stockfish-PVS cannot be used with the unpatched runtime because of a rollback bug. Only use it with the modchip binary.

Licence and Safety Notes

• Tooling: Apache 2.0, so commercial use is fine with attribution.
• Weights: MIT, inherited from DeepSeek V4-Flash.
• Privacy: everything runs on your machine, and no prompts leave it.
• Binaries: the pre-built executables come from a weeks-old project with one main author. Building from source or running in a container is the cautious choice.
• Claims: the “world’s first” tagline is marketing, and all speed and quality figures are self-reported.

My Take

This is the most exciting local-AI hack of the month, with a big asterisk. Getting a 284B frontier-class MoE to answer at 10 tok/s on a laptop CPU shows how far MoE sparsity plus clever kernels can go.

But a 64 GB PC is the absolute edge, not the sweet spot. On a 96-128 GB machine like my own rig, this build becomes genuinely usable, with room left for a browser and an editor. I would wait for independent coding and reasoning benchmarks before trusting a 1.71-bit model with real work.

If you have 16-32 GB, the 30B Nemotron builds are the practical win here. Fast CPU-only inference at 25-60 tok/s with no GPU is useful today.

FAQ

Can I really run DeepSeek 284B without a GPU?

According to the developer, yes: about 9-15 tok/s on a 16-core Zen 5 CPU with 64 GB DDR5. Nobody has independently confirmed it yet.

Is this the full DeepSeek model?

It has all 284B parameters, but compressed to 1.71 bits per weight on average. Some quality loss versus the full-precision model is expected, though not yet measured.

Does it work in Ollama or LM Studio?

No. The custom Fractal-64 tensors cause an “unknown tensor type 40” error. You need the llama-modchip binary.

Will it run on 32 GB of RAM?

Not the 284B model. On 32 GB, run Nemotron-3.5-30B-A3B or Qwen3-30B-A3B builds from the same project.

Is it DeepSeek V4.1 Flash?

No, it is the 284B V4-Flash-0731 from July. V4.1 Flash is much larger and needs 128 GB or more.

Does it work on Windows?

There is a Windows launcher and a hybrid GPU+CPU pack, but 284B results on Windows have not been published. Linux is the safest choice.

Is it free?

Yes. The tools are Apache 2.0 and the weights are MIT, so there is nothing to pay.

Sources

• synchopate/llama-modchip on GitHub: README v2.4.0 and release notes v2.1.0-v2.4.0 (2-4 Nov 2026)
• Hugging Face model card: Synchopate/DeepSeek-V4-Flash-0731-Fractal-64GB-CPU-GGUF (files, quant mix, commands, baseline speeds)
• Windows Forum: “llama-modchip Claims DeepSeek V4 Flash 284B Runs on 64GB RAM Without a GPU” (8 Nov 2026)
• CuriousLM: “llama-modchip Squeezes 284B DeepSeek V4-Flash Into 64GB of RAM, No GPU” (9 Nov 2026)
• AI Weekly alert: “llama-modchip runs 284B DeepSeek V4-Flash on 64GB CPU box” (9 Nov 2026)
• noze.it: “DeepSeek V4 Flash 0731: same architecture, new post-training” (specs, benchmarks, API prices)
• OpenRouter: DeepSeek V4 Flash 20260731 model page (context, pricing, release date)