ComfyUI-GGUF Guide: Install, Unet Loader (GGUF), fp8 vs GGUF and Fixes (2026)

ComfyUI-GGUF is the custom node pack that lets ComfyUI load GGUF files, the same compressed model format that llama.cpp and Ollama use. With it you can run big image and video models such as Wan 2.2, Qwen-Image, FLUX and LTX on 8, 12 or 16 GB GPUs by loading a Q4, Q5, Q6 or Q8 version of the model instead of the full 20-40 GB safetensors file. This guide shows how to install ComfyUI-GGUF on Windows portable, Desktop and Linux, where GGUF files go, how to use Unet Loader (GGUF) and the GGUF CLIP loaders, how GGUF compares with fp8, and how to fix the errors people hit most often.

What ComfyUI-GGUF Does

Out of the box, ComfyUI loads diffusion models from .safetensors files in bf16, fp16, fp8 or newer formats such as NVFP4 and int8. GGUF is a different container that stores weights in “k-quant” formats like Q4_K_M, Q5_K_M, Q6_K and Q8_0. Those formats were designed for LLMs in llama.cpp, but they work just as well for the large transformer models that modern image and video generators use.

The ComfyUI-GGUF pack, written by city96, adds loader nodes that read GGUF files and dequantize the weights on the fly while the model runs. The practical result:

• A 14B video model that is about 28 GB in bf16 can load from a GGUF file of roughly 8-9 GB at Q4_K_M or about 15 GB at Q8_0.
• You keep using the rest of your workflow unchanged: same samplers, same VAE, same LoRA nodes.
• It works on any GPU that runs ComfyUI, because GGUF weights are turned back into your normal compute precision before each matrix multiply. There is no “your GPU does not support fp8” problem.

If you are new to quant names, my GGUF quantization explained guide covers what Q4_K_M, Q6_K and Q8_0 actually mean.

How to Install ComfyUI-GGUF

Option 1: ComfyUI-Manager (easiest)

1. Open ComfyUI and click Manager (or the puzzle icon in the new UI).
2. Choose Custom Nodes Manager and search for ComfyUI-GGUF. Pick the one by city96.
3. Click Install, then restart ComfyUI when asked.

If you open a workflow that already uses GGUF nodes, Manager’s Install Missing Custom Nodes button will find it for you.

Option 2: Windows portable, manual

Open a terminal in the folder that contains run_nvidia_gpu.bat and run:

git clone https://github.com/city96/ComfyUI-GGUF ComfyUI/custom_nodes/ComfyUI-GGUF
.\python_embeded\python.exe -s -m pip install -r .\ComfyUI\custom_nodes\ComfyUI-GGUF\requirements.txt

The second line is important. The portable build has its own Python in python_embeded, and installing gguf with your system pip does nothing for ComfyUI.

Option 3: Linux, macOS or a venv install

cd ComfyUI/custom_nodes
git clone https://github.com/city96/ComfyUI-GGUF
# activate the same venv/conda env you start ComfyUI with, then:
pip install --upgrade gguf

On ComfyUI Desktop, use the built-in terminal (or Manager) so the package lands in the app’s own environment. Restart ComfyUI after any install.

Updating

Use Manager > Update, or run git pull inside ComfyUI/custom_nodes/ComfyUI-GGUF. New model families (a new video model, a new text encoder) often need an updated pack, so update it before you try a freshly released GGUF.

The GGUF Nodes and What They Replace

You will find them by double-clicking the canvas and typing “GGUF”, or in the node menu under bootleg. The GGUF CLIP loaders accept both GGUF and regular safetensors files, so you can mix, for example, a GGUF T5 with a normal CLIP-L in a DualCLIPLoader (GGUF).

What does not change: the Load VAE node, LoRA loaders (LoraLoaderModelOnly works on a GGUF model; the README calls LoRA support experimental but it works for most models), samplers and everything after them.

Where to Put GGUF Files

Never put GGUF files in models/checkpoints/; that folder is only for all-in-one checkpoints. On Windows portable the full path starts with ComfyUI_windows_portable\ComfyUI\models\. Every other folder is listed in my ComfyUI model folders cheat sheet.

Using Unet Loader (GGUF) Step by Step

1. Open the normal workflow for your model, ideally the official template from Workflow > Browse Templates. It will have a Load Diffusion Model node.
2. Add Unet Loader (GGUF): double-click the canvas, type Unet Loader (GGUF) and add it.
3. Pick your file in its unet_name list. If the list is empty, press R to refresh, and check the file really ends in .gguf and sits in unet/ or diffusion_models/.
4. Rewire: drag the purple MODEL output from the GGUF loader to everything the old loader fed (often a LoRA loader or ModelSamplingSD3/AuraFlow node), then delete the old loader.
5. Optional, text encoder: replace Load CLIP with CLIPLoader (GGUF), select the GGUF encoder and set the same type the old node used (for example wan, qwen_image, flux or ltxv).
6. Run. The first run is slower while the file is read from disk; later runs reuse it from RAM.

For video models with two experts, such as Wan 2.2 14B (high-noise and low-noise), you need two Unet Loader (GGUF) nodes, one for each file. My Wan 2.2 on 8 to 16 GB guide shows that exact layout.

fp8 vs GGUF: Which Should You Use?

This is the most common question once ComfyUI-GGUF is installed. The honest answer is that it depends on whether the fp8 file fits.

Rule of thumb:

• If the fp8 (or int8/NVFP4) file fits in your VRAM with some room to spare, use it. It is the simplest and usually the fastest.
• If it does not fit, or you have only 8-12 GB, use GGUF and pick the largest quant that fits (see the next section).
• If you have plenty of system RAM and a fast NVMe drive, try both. Recent ComfyUI versions with dynamic VRAM handle offloading much better than a year ago, and on some machines int8 or fp8 with offloading now matches Q4_K_M speed. ComfyUI’s own advice is to keep dynamic VRAM enabled and prefer native formats where they work.

Which GGUF Quant for Your VRAM

File size is roughly parameters x bits per weight / 8. Real files are a little bigger because some layers are kept at higher precision. These are estimates for planning, not measurements:

Also leave room for the text encoder, the VAE decode and the latent for your resolution and frame count. For long video, drop one quant level before you drop resolution. My how much VRAM for ComfyUI post explains how ComfyUI splits memory between these parts.

Popular GGUF Models for ComfyUI

Always use a GGUF made for the exact model version in your workflow. A Qwen-Image-Edit 2509 GGUF will not behave like 2511, and a Wan 2.1 GGUF will not work as a Wan 2.2 expert.

Common ComfyUI-GGUF Errors and Fixes

“node types were not found: UnetLoaderGGUF”

The pack is not installed or failed to load. Install it (see above), then read the ComfyUI startup log for IMPORT FAILED next to ComfyUI-GGUF. The usual cause is that gguf was installed into a different Python than the one ComfyUI runs on.

“ModuleNotFoundError: No module named ‘gguf’”

Same cause. On Windows portable run .\python_embeded\python.exe -s -m pip install -r .\ComfyUI\custom_nodes\ComfyUI-GGUF\requirements.txt from the portable root. In a venv, activate it first, then pip install --upgrade gguf. Restart ComfyUI.

Unet Loader (GGUF) shows no files

The file is in the wrong folder (for example checkpoints/), is still downloading (a .part or .crdownload file), or ComfyUI has not refreshed. Move it to models/unet/ or models/diffusion_models/ and press R. If you use extra_model_paths.yaml, make sure the unet or diffusion_models entry points at the right drive.

“ERROR: Could not detect model type of: …”

The loader cannot recognise the architecture from the tensor names. Either the file is truncated (compare its size with the Hugging Face page and re-download), it is an LLM GGUF rather than a diffusion model, or your ComfyUI-GGUF and ComfyUI are too old for a new model family. Update both.

“ERROR UNSUPPORTED UNET”

The file was read, but that model variant is not supported by your version of the pack. Update ComfyUI-GGUF and ComfyUI; if it is a brand-new model, wait for support or use the safetensors version.

“Mixing scaled FP8 with GGUF is not supported!”

You loaded a scaled fp8 text encoder through a GGUF CLIP loader together with GGUF files, or the other way round. Keep one format for the whole text encoder path: either all GGUF in the GGUF CLIP loader, or the fp8 file in the regular Load CLIP node.

Edit model ignores the input image or errors at the text encoder

Vision-language text encoders (Qwen2.5-VL for Qwen-Image-Edit, for example) need the matching mmproj GGUF next to the main encoder file in text_encoders/. Without it, the image part of the prompt fails. Download the mmproj from the same repo as the encoder.

Tokenizer errors with GGUF text encoders

Some GGUF encoders need sentencepiece and protobuf. Install both into ComfyUI’s Python the same way as gguf.

“Error while deserializing header: HeaderTooLarge”

Not a GGUF problem: a safetensors file in the workflow is incomplete or is a tiny Git LFS pointer. Re-download it and check the size.

ComfyUI closes or shows “Reconnecting” while loading a GGUF

System RAM ran out while reading the file. Use a smaller quant, close other apps and enlarge the Windows page file. Full steps are in my ComfyUI Reconnecting fix, and other messages are in the ComfyUI errors and fixes list.

macOS: “buffer is not large enough”

Some PyTorch nightly builds on Apple silicon trigger this. The ComfyUI-GGUF README points to issue #107 for workarounds; using a stable PyTorch release usually helps.

Frequently Asked Questions

What is ComfyUI-GGUF?

ComfyUI-GGUF is a custom node pack by city96 that adds GGUF loaders to ComfyUI. It lets you load quantized Q4, Q5, Q6 and Q8 versions of diffusion models and text encoders so large models fit on smaller GPUs.

Where do I put GGUF models in ComfyUI?

Put GGUF diffusion models in ComfyUI/models/unet/ or ComfyUI/models/diffusion_models/ and GGUF text encoders in ComfyUI/models/text_encoders/ or ComfyUI/models/clip/. Do not put them in models/checkpoints/.

What is the difference between Unet Loader (GGUF) and Load Diffusion Model?

Load Diffusion Model reads safetensors files only. Unet Loader (GGUF) reads .gguf files and dequantizes them while the model runs. Both output the same MODEL connection, so the rest of the workflow stays the same.

Is fp8 or GGUF better in ComfyUI?

If the fp8 file fits in your VRAM, fp8 is usually faster with very similar quality. If it does not fit, a GGUF such as Q8_0, Q6_K or Q4_K_M is usually the better choice because it avoids heavy offloading and works on any GPU.

Can I use LoRAs with GGUF models?

Yes. Connect the GGUF model output to a normal LoraLoaderModelOnly node. The ComfyUI-GGUF README describes LoRA support as experimental, but it works with most popular models.

Which GGUF quant should I download?

Pick the largest quant that fits your VRAM with room for the text encoder and VAE. Q8_0 is nearly lossless, Q6_K and Q5_K_M are very good, and Q4_K_M is the usual choice for 8-12 GB GPUs.

Why does Unet Loader (GGUF) show no models?

The file is in the wrong folder, is still downloading, or the list was not refreshed. Move the .gguf file to models/unet or models/diffusion_models and press R in ComfyUI to refresh.

Final Thoughts

ComfyUI-GGUF is still the easiest way to run 14B and 20B image and video models on an 8-16 GB GPU. Install it with Manager, make sure gguf is in ComfyUI’s own Python, drop GGUF files into models/unet and models/text_encoders, and swap the stock loaders for Unet Loader (GGUF) and CLIPLoader (GGUF). If the fp8 file fits your card, use fp8; if not, take the largest GGUF quant that fits.

Next steps: set up Wan 2.2 with GGUF on 16 GB, try LTX-2.5 GGUF, or get every Qwen-Image model file in the right place. If you also run LLMs, the same GGUF files power Ollama, LM Studio and llama.cpp.

Sources

• GitHub: city96/ComfyUI-GGUF (README, install steps, nodes)
• Comfy Registry: ComfyUI-GGUF
• GitHub issue #463: GGUF vs native formats and dynamic VRAM
• GitHub issue #416: Unet Loader (GGUF) shows no options
• LocalAIMaster: ComfyUI quant load errors (GGUF, fp8, NVFP4, INT8)
• Hugging Face: city96/FLUX.1-dev-gguf
• Hugging Face: QuantStack/Qwen-Image-GGUF