How Much VRAM Do I Need for AI? LLM, Image and Video VRAM Requirements (2026)

How much VRAM do you need for AI? It depends on three things: the size of the model, how heavily it is compressed (quantized), and how much context or resolution you ask for. The good news is that you can work it out in a minute with simple maths, without buying a card first and hoping. This guide gives you the formulas, ready-made tables for LLMs from 4B to 120B, and real requirements for image and video models like FLUX 3 Klein, Qwen-Image 2.1, Wan 2.2 and LTX-2.5, based on the setups I run and the files the model makers publish.

The VRAM Formula for LLMs

A model’s weights are just numbers. A 7-billion-parameter model stored at 16 bits per number takes 7 x 16 / 8 = 14 GB. Quantization stores each number in fewer bits, so the same model at about 4.8 bits (GGUF Q4_K_M) takes about 4.2 GB, plus a little for embeddings and metadata. The full formula is:

Total VRAM = weights + KV cache + runtime overhead

• Weights (GB) = parameters (billions) x bits per weight / 8
• KV cache = memory the model uses to remember your conversation (see below)
• Overhead = CUDA context, compute buffers and the desktop if this is your display GPU: budget 0.5-1.5 GB

The effective bits per weight for common GGUF files are roughly: Q4_K_M 4.8, Q5_K_M 5.7, Q6_K 6.6, Q8_0 8.5 and F16 16. My GGUF quantization guide explains the quality trade-off of each. The short version: Q4_K_M is the sweet spot for most people, Q5/Q6 if you have room, and Q8 is nearly lossless.

LLM VRAM Table: 4B to 120B

Weight size only, rounded. Add the KV cache and about 1 GB of overhead to get what you need on the card.

Real files differ a little because some layers are kept at higher precision, so always check the file size on the Hugging Face or Ollama page. For example, Llama 3.3 70B at Q4_K_M is 42.5 GB, which is why two 24 GB cards are the classic 70B setup.

VRAM for Popular Models Right Now (November 2026)

Most people searching “how much VRAM” have one model in mind. Here are real download sizes for the models people ask me about most, taken from the GGUF and Ollama pages I used in each setup guide. Add 1-3 GB for context and overhead.

Notice the pattern: for MoE models the question is not “how much VRAM” but “how much total memory”. Kimi K3 needs hundreds of gigabytes no matter how fast your GPU is, while a 30B MoE like GLM-4.7-Flash runs fine on a 16 GB card because most of its weights can sit in system RAM.

Worked Example: Will It Fit on My GPU?

Say you have a 16 GB card and want Qwen3.8-27B with a 16K context.

1. Weights: 27B x 4.8 bits / 8 = about 16.2 GB at Q4_K_M (the real file is 17.1 GB). Already over 16 GB.
2. Try a smaller quant: the UD-Q3_K_XL file is 13.4 GB.
3. Context: Qwen3.8-27B only uses full attention in 16 of its 64 layers, so 16K of context costs roughly 1 GB, far less than an older 27B model.
4. Overhead: about 1 GB for CUDA and buffers. Use the integrated GPU for your monitor if you can.
5. Total: 13.4 + 1 + 1 = about 15.4 GB. It fits, with a little room to spare.

If the total is slightly over, you have three options: drop one quant level, cut the context, or offload a few layers to the CPU (slower, but it works). Check the result with ollama ps, which shows the GPU/CPU split. Anything under 100% GPU means part of the model is running on the CPU.

Context Length and the KV Cache

Every token in your conversation stores a key and value vector in every layer. The size per token is:

KV bytes per token = 2 x layers x KV heads x head dimension x bytes per value

Llama 3.1 8B has 32 layers, 8 KV heads and a head dimension of 128. At 16-bit that is 2 x 32 x 8 x 128 x 2 = 128 KB per token. So an 8K context costs 1 GB and a 32K context costs 4 GB, on top of the 4.9 GB of Q4 weights. A 70B model with 80 layers needs about 320 KB per token, so 32K of context adds about 10 GB.

Three ways to shrink it:

• Lower the context. In Ollama set num_ctx (or OLLAMA_CONTEXT_LENGTH), in llama.cpp use -c 16384. Many people run 128K context they never use.
• Quantize the KV cache. llama.cpp supports --cache-type-k q8_0 --cache-type-v q8_0, which halves the cache with almost no quality loss. In Ollama set OLLAMA_KV_CACHE_TYPE=q8_0 together with OLLAMA_FLASH_ATTENTION=1.
• Pick models with efficient attention. Newer models with sliding-window or hybrid linear attention (Gemma, Qwen3.5/3.8 hybrids) need far less cache per token than older designs.

All of these settings are listed in my Ollama commands cheat sheet.

MoE Models and RAM Offload

Mixture-of-experts models change the maths. A model like Qwen3.5-35B-A3B has 35 billion parameters in total but only about 3 billion are used for each token. You still need memory for all 35B, but each token only reads a small part of it, so the model stays fast even when most of it sits in system RAM. That is why a 16 GB GPU plus 64 GB of DDR5 can run 100B+ MoE models at usable speeds, while a dense 70B model offloaded the same way crawls at 1-3 tokens/s.

In llama.cpp the trick is to keep attention and shared layers on the GPU and push the expert weights to the CPU with --n-cpu-moe (or the older -ot override-tensor flag). LM Studio and Ollama do a simpler version automatically. On my own machine (2x RTX 5080 16 GB and 128 GB RAM) this is how I run Qwen3.8-Flash-Next, a 125B MoE with about 6B active parameters.

The catch: system RAM is now expensive too. A 64 GB DDR5 kit costs over $900 in late 2026, more than double what it cost a year ago. I cover why in why RAM is so expensive in 2026.

VRAM for Image Generation

Image models are judged differently: the diffusion model, the text encoder and the VAE all need memory, but ComfyUI loads and unloads them in turn, and resolution adds working memory. These are realistic minimums for 1024×1024 in ComfyUI:

The text encoders are often the hidden cost. The original Qwen-Image uses a 7B vision-language model (Qwen2.5-VL 7B) as its text encoder, which is 8-16 GB by itself at fp8/fp16. ComfyUI can run text encoders on the CPU or a second GPU to save VRAM, and my Qwen-Image model files guide lists which encoder file to use.

VRAM for Video Generation

Video is the most memory-hungry local AI job because the model processes every frame together. Resolution and frame count multiply the working memory.

For video, 24-32 GB is where things stop being a fight. With 16 GB you can do almost everything, but you will be using GGUF files, block swapping (moving transformer blocks to RAM between steps) and tiled VAE decoding, which all cost time. 64 GB or more of system RAM helps a lot because offloaded blocks need somewhere to go.

VRAM for LoRA Training and Fine-Tuning

Training needs much more memory than inference because it also stores gradients and optimizer states. Rough guide:

• SDXL or small image model LoRA: 8-12 GB.
• Image LoRA on modern 6-20B models: 6-24 GB depending on tricks. KohakuMoss trains MOSS-Image LoRAs in about 6 GB (see MOSS LoRA training).
• QLoRA on an 8B LLM: about 8-12 GB. On a 70B LLM: about 48 GB.
• Full fine-tune of an 8B LLM: 60-80 GB+ (multiple GPUs or a rented H100).

For anything above a LoRA, renting is usually cheaper than buying. See the cheapest cloud GPUs for AI.

What Each VRAM Size Can Do

If you are shopping, my best GPU for local LLMs guide turns this into specific cards with November 2026 prices.

Apple Silicon, AMD and Multi-GPU: How Memory Works There

• Mac (M-series): CPU and GPU share one pool of unified memory. macOS keeps some for itself, so by default plan on using roughly 65-75% of total RAM for models (a 64 GB Mac gives about 45 GB for a model). That is why 128 GB and 512 GB Mac Studios are popular for very large MoE models. Speed is lower than an NVIDIA card of the same capacity. See Mac vs PC for local AI.
• AMD Radeon: the same VRAM maths applies. llama.cpp, Ollama and LM Studio support ROCm and Vulkan; ComfyUI support is improving but some custom nodes are CUDA-only.
• Two GPUs: for LLMs, VRAM adds up. llama.cpp and Ollama split layers across cards, so 2x 16 GB behaves like about 30 GB (each card keeps a small buffer). For image and video in ComfyUI, VRAM does not pool: a model must fit on one card, but you can put the text encoder or VAE on the second card with multi-GPU nodes.
• Integrated GPUs and NPUs: laptops with shared memory can run small models, but speed is limited by memory bandwidth. NPUs are mostly for small on-device features; see what is an NPU.

Out of Memory? Fixes by Error Message

More ComfyUI-specific fixes are in my ComfyUI errors and fixes guide.

How to Use Less VRAM

• Use a smaller quant (Q4_K_M instead of Q8_0) or a GGUF version of image and video models.
• Cut the context to what you actually use and quantize the KV cache.
• Run your monitor from the integrated GPU so the desktop does not eat 0.5-1 GB of VRAM.
• In ComfyUI, use --lowvram or --reserve-vram, tiled VAE decode, and load text encoders on the CPU. All flags are in my ComfyUI command line arguments list.
• Prefer MoE models when you have lots of system RAM and little VRAM.

Frequently Asked Questions

How much VRAM do I need to run a 7B or 8B model?

About 5 GB for the Q4_K_M weights plus 1-2 GB for context and overhead, so an 8 GB GPU is enough. At Q8_0 you need about 10-11 GB.

How much VRAM do I need for a 70B model?

About 42.5 GB for Q4_K_M weights plus 5-10 GB for context, so 48 GB total (two 24 GB cards) is the practical minimum for full GPU speed.

Is 8 GB of VRAM enough for AI in 2026?

It is enough to learn and to run 7B-9B models and SDXL. For modern image and video models you will rely on GGUF and offload, which works but is slow. 16 GB is a much better starting point.

Does system RAM count as VRAM?

No, but models can spill into it. Dense models slow down a lot when they do; MoE models handle it well. Integrated GPUs and Apple Silicon share system memory, so there it does count.

How do I check how much VRAM a model is using?

Run nvidia-smi on NVIDIA, or ollama ps to see the CPU/GPU split of a loaded Ollama model.

Final Thoughts

The formula is simple: about 0.6 GB per billion parameters at Q4, plus context, plus a gigabyte of headroom. Use it before you download a model or buy a GPU, and you will never be surprised by an out-of-memory error again. If you already hit one in ComfyUI, my ComfyUI errors and fixes list covers it.

Sources

• llama.cpp (GGUF quant types, KV cache types and –n-cpu-moe)
• Llama 3.1 8B config (layers, KV heads, head dimension)
• Ollama FAQ (context length, flash attention, KV cache type)
• OpenClaw: Llama 3.3 70B Q4_K_M file size on 2x 24 GB
• Engadget: RAM prices 2020 vs 2026 (2 Nov 2026)