How to Use GGUF Models in ComfyUI: Install, Folders, Quants and Fixes (2026)

Big image and video models like Flux, Wan, Qwen-Image and MiniMax H3 can need 20 GB or more of VRAM in their full form. GGUF versions shrink them so they run on 8 GB, 12 GB and 16 GB cards.

ComfyUI cannot load GGUF files on its own. You need one small custom node pack, the right folders and the right loader node, and then a GGUF model works in almost any workflow.

This guide shows you how to use GGUF models in ComfyUI step by step: installing ComfyUI-GGUF, where to put the files, how to swap the loaders, which quant to pick for your VRAM, GGUF text encoders, LoRAs, and how to fix the common errors.

What GGUF Means in ComfyUI

GGUF is a file format made popular by llama.cpp for running language models with fewer bits per weight. The ComfyUI-GGUF project brought the same trick to image and video models.

Here is the key point from its author: quantization does not work well for old convolution-based UNET models, but transformer (DiT) models such as Flux are much less affected. That is why GGUF is popular for Flux, Wan, Qwen-Image and other modern models, and rarely used for SD 1.5 or SDXL.

A GGUF quant trades a little quality for a lot less memory. For example, city96’s Flux.1 Dev files range from 23.8 GB (F16) down to 12.7 GB (Q8_0), 6.81 GB (Q4_K_S) and 4.03 GB (Q2_K).

Which Models Work as GGUF

The loader checks each file’s architecture against a list. If the architecture is not on it, you get an “Unexpected architecture type” error. Here is what each version of the node pack accepts right now:

The two packs share the same node names, so a workflow made for one opens with the other. Install only one of them at a time. If you mostly run Flux, Wan or the first Qwen-Image, the original city96 pack is fine.

If you want MiniMax H3 or Qwen-Image 2.1 GGUFs, install the leejet fork instead.

Step 1: Install ComfyUI-GGUF

Update ComfyUI first. The README warns that older ComfyUI builds lack the custom ops the loader needs. Our guide to updating ComfyUI safely covers Desktop, Portable and Git installs.

Option A: ComfyUI Manager (easiest)

1. Open ComfyUI and click Manager (new UI: the puzzle-piece Extensions button, then Custom Nodes Manager).
2. Search for GGUF and pick ComfyUI-GGUF by city96.
3. Click Install, then Restart when asked.
4. Refresh the browser so the new nodes appear.

No Manager yet? See how to install ComfyUI Manager. For the leejet fork, use Manager’s Install via Git URL with https://github.com/leejet/ComfyUI-GGUF.

Option B: Git (manual install or Linux)

cd ComfyUI/custom_nodes
git clone https://github.com/city96/ComfyUI-GGUF
pip install --upgrade gguf

Run pip inside the same Python environment that runs ComfyUI (activate your venv or conda env first).

Option C: Windows Portable

Open a Command Prompt in the ComfyUI_windows_portable folder (where run_nvidia_gpu.bat lives) and run:

git clone https://github.com/city96/ComfyUI-GGUF ComfyUI/custom_nodes/ComfyUI-GGUF
.\python_embeded\python.exe -s -m pip install -r .\ComfyUI\custom_nodes\ComfyUI-GGUF\requirements.txt

The requirements file installs gguf>=0.13.0, plus sentencepiece and protobuf for GGUF text encoders.

Step 2: Download and Place GGUF Files

GGUF models live on Hugging Face. Good sources are city96 (Flux, SD3.5, Qwen-Image, T5), QuantStack (Wan and others) and the model makers’ own GGUF repos. Download one quant only; you do not need every file in the repo.

Comfy Desktop: the models folder is inside the base path you picked at install. Open it from the app menu or check extra_models_config.yaml if you share models between installs.

After copying files, press R in ComfyUI (or click Refresh) so the dropdowns pick them up.

Step 3: Swap the Loader in Your Workflow

You do not need a special GGUF workflow. Take the normal template for your model and change one or two nodes.

1. Open your usual workflow, for example the Flux, Wan or Qwen-Image template from Workflow > Browse Templates.
2. Double-click the canvas, search Unet Loader (GGUF) and add it. It sits in the bootleg category.
3. Pick your .gguf file in its unet_name dropdown.
4. Drag its MODEL output to wherever the old “Load Diffusion Model” connected (usually a LoRA loader, ModelSamplingFlux/SD3 or the KSampler).
5. Delete the old loader.
6. Optional: replace the text encoder loader with the GGUF version (see below) if you downloaded a GGUF encoder.
7. Click Run. The console shows the GGUF file loading, and VRAM use should be much lower than before.

For Wan 2.2 14B, which uses two experts (high noise and low noise), add two Unet Loader (GGUF) nodes, one per file, in place of the two normal loaders. Our ComfyUI image-to-video guide shows that workflow in full.

A worked example with Qwen-Image is in our Qwen-Image 2.1 GGUF on 8 GB and 12 GB guide.

Which Quant to Pick for Your VRAM

The number after Q is roughly the bits per weight. Lower is smaller and faster to load, but quality drops, slowly at first and then quickly below Q4.

Remember that the model is not the only thing in VRAM. The text encoder, VAE, LoRAs and the latent itself also need room, and video models need far more for each extra frame. ComfyUI can offload parts to system RAM, so a slightly too large model still runs, only slower.

The loader also handles the K-quants (Q2_K to Q6_K), Q4_0/Q4_1, Q5_0/Q5_1, Q8_0, BF16 and the IQ4_NL and IQ4_XS types. If you have a choice, K-quants (the ones with K in the name) usually look better than the old _0/_1 quants at the same size.

Still unsure what your card can handle? Our RTX 3060 12 GB for AI and RTX 5060 Ti 16 GB for AI guides list real models that fit.

GGUF Text Encoders (T5, Qwen, Gemma)

Text encoders can be huge too. Flux uses T5-XXL, Wan uses UMT5-XXL and Qwen-Image uses the Qwen2.5-VL language model as its encoder. ComfyUI-GGUF includes GGUF versions of the CLIP loaders:

• CLIPLoader (GGUF): one encoder (Qwen-Image, Wan, Lumina 2 and similar).
• DualCLIPLoader (GGUF): two encoders, such as Flux (clip_l + T5).
• TripleCLIPLoader (GGUF) and QuadrupleCLIPLoader (GGUF): SD3.5 and HiDream.

These loaders accept both .gguf and normal .safetensors files, so you can mix a GGUF T5 with the regular clip_l. Set the type dropdown to match your model (for example flux), just like the stock loader.

The text encoder only runs once per prompt, so a Q5 or Q6 encoder saves memory with very little visible change. It is often the easiest win on 8 GB cards.

Using LoRAs with GGUF Models

LoRAs work with GGUF models through the normal LoraLoaderModelOnly or Load LoRA nodes. Connect the Unet Loader (GGUF) MODEL output into the LoRA loader, then onward to the sampler.

The README still calls LoRA support experimental. In practice most Flux and Wan LoRAs work, but each LoRA adds some VRAM and can slow the first step while weights are patched. If a LoRA has no effect, check it was trained for the same base model.

Unet Loader (GGUF/Advanced) Settings

The Advanced loader adds three options. Leave them on default unless you are fixing a problem.

On older GPUs without fast bfloat16 (GTX 10 and 16 series), setting dequant_dtype to float16 can fix black images or errors.

Make Your Own GGUF (Optional)

Most popular models already have GGUF uploads, but the repo includes tools to quantize your own:

1. Convert to F16/BF16 GGUF: python convert.py --src E:\models\unet\model.safetensors (needs gguf>=0.13.0).
2. Build a patched llama-quantize: clone llama.cpp, check out the tag named in the tools README, apply the included lcpp.patch, and compile the quantize target.
3. Quantize: run llama-quantize on the F16 file with a type such as Q5_K_S.
4. For Wan and HunyuanVideo, run fix_5d_tensor.py afterwards, as the tools README explains, or the file will not load.

Use the reference checkpoint format, not the diffusers format, for Flux. Save it through ComfyUI’s ModelSave node first if needed.

Troubleshooting

For general memory errors, our ComfyUI inpainting fix guide also covers the usual VRAM checks.

My Take

GGUF is the single biggest reason modern models run on mid-range cards. My rule of thumb: use Q8_0 if it fits, drop to Q6_K or Q5_K_S on 12 GB cards, and use Q4_K_S only on 8 GB. Pair it with a GGUF text encoder before going any lower.

Install one pack only. For most people that is city96’s original. If you run MiniMax H3 or the newest Qwen-Image builds, switch to the leejet fork, and keep an eye on the original repo, since support often lands there later.

FAQ

Can ComfyUI load GGUF without custom nodes?

No. ComfyUI’s built-in loaders read safetensors and checkpoints. You need ComfyUI-GGUF (or its fork) for .gguf files.

Where do I put GGUF files in ComfyUI?

Diffusion models go in ComfyUI/models/diffusion_models or models/unet. GGUF text encoders go in models/text_encoders or models/clip.

Is GGUF slower than safetensors?

Usually a little, because weights are unpacked on the fly. But if the full model does not fit in VRAM, GGUF is much faster than offloading a big safetensors file to system RAM.

Which is better, Q4_K_S or Q4_0?

They are about the same size. Q4_K_S generally keeps a bit more detail, so pick it when both are available.

Can I use LLM GGUF files from Ollama or LM Studio in ComfyUI?

Only as text encoders, and only for supported types (T5, Llama, Qwen, Gemma 3). Chat model GGUFs are not image models, so they will not work in the Unet loader.

Do LoRAs work with GGUF models?

Yes, with the normal LoRA loader nodes. Support is marked experimental, and each LoRA adds some memory use.

Should I install both city96 and leejet versions?

No. They register the same node names and will clash. Install one, and remove the other folder from custom_nodes.

Sources

• city96/ComfyUI-GGUF (README, nodes and supported architectures)
• leejet/ComfyUI-GGUF fork (MiniMax, Krea 2, Ideogram 4, Qwen-Image 2.1 support)
• ComfyUI-GGUF tools: converting and quantizing models
• city96/FLUX.1-dev-gguf (quant file sizes)
• city96 T5-XXL encoder GGUF