How to Run Qwen3.8-27B on a 16 GB GPU (2026): Best GGUF, Ollama, LM Studio and llama.cpp Settings

Qwen3.8-27B is the model I recommend most often for local use in late 2026: a dense 27B with vision, a 256K context and coding scores that Alibaba puts ahead of some frontier cloud models. The catch is size.

Ollama’s default download is 18 GB, which is just too big for the most common enthusiast card, the 16 GB RTX 4060 Ti, 4070 Ti Super, 4080, 5070 Ti or 5080.

This guide shows exactly how to run Qwen3.8-27B on a 16 GB GPU: which GGUF to pick, how much context you can afford, and the right settings for Ollama, LM Studio and llama.cpp.

I also cover 12 GB, 24 GB and two-GPU setups like my own 2x RTX 5080.

What Qwen3.8-27B Is

Alibaba’s Qwen team released Qwen3.8-27B on 14 August 2026 under the Apache 2.0 licence, with the weights on Hugging Face (Qwen/Qwen3.8-27B and an official Qwen/Qwen3.8-27B-FP8) and ModelScope. The key facts:

• Dense, not mixture-of-experts. Every token uses all 27B language-model parameters (27.78B in total including the vision encoder).
• 64 layers in a hybrid pattern: three Gated DeltaNet (linear attention) blocks for every one Gated Attention block, so only 16 layers keep a normal KV cache.
• 262,144-token native context, which Alibaba says can be stretched to 1M with YaRN (with some accuracy loss on short inputs in some frameworks).
• Native vision-language model: it takes text, images and video and answers in text.
• Multi-token prediction in training, and a thinking mode that is on by default.

It is the open-weight member of the Qwen3.8 generation. Its much larger sibling, the Qwen3.8-Flash-Next MoE, needs a big RAM offload setup; this article is about the 27B that fits normal gaming PCs.

How Good Is It? Official Benchmarks

Alibaba’s own results, as reported by DataNorth. Alibaba ran them through the Claude Code harness at temperature 1.0 with a 256K window and modified some benchmarks, and two of them are in-house, so treat this as the vendor’s view:

Even allowing for vendor bias, it is a big step over Qwen3.6-27B (Alibaba lists DeepSWE 1.1 at 42.2% vs 13.3%), and in my best Ollama models ranking it is the best all-round local model for 16-24 GB.

Why a 27B Model Can Work on 16 GB

Memory for a local LLM has two parts: the weights (fixed, set by the quant you download) and the KV cache (grows with context length). Two things help Qwen3.8-27B here:

1. Good low-bit quants. Unsloth’s Dynamic quants keep the most sensitive tensors at higher precision, so a 3-bit file at 13.4 GB holds up much better than a plain 3-bit file would.
2. Hybrid attention. The Gated DeltaNet layers keep a small fixed-size state instead of a cache that grows with every token. Only the 16 full-attention layers have a growing KV cache, so a long context costs roughly a quarter of what an all-attention 27B model would need.

On a 16 GB card, Windows and your display already take a few hundred MB, so plan on roughly 15 GB usable.

A 13.4 GB model plus the 0.93 GB vision projector leaves little room, which is why you either drop the projector for text-only use, quantize the KV cache, or keep the context moderate.

If the weights and context do not fit, part of the model runs on the CPU. Because this is a dense model, every token passes through those CPU layers, so even a small offload slows generation noticeably. My GGUF quantization guide explains the maths.

Which GGUF to Download for Your VRAM

File sizes from unsloth/Qwen3.8-27B-GGUF (Unsloth Dynamic v3 preview):

My recommendation for 16 GB: use UD-Q3_K_XL fully on the GPU for interactive chat and coding, because speed matters when you are waiting for each answer. Use Q4_K_M with a small offload for batch jobs or hard questions where you do not mind waiting.

Run Qwen3.8-27B With Ollama

The simplest option is Ollama’s own build (18 GB). On a 16 GB card Ollama automatically places what does not fit on the CPU:

ollama run qwen3.8:27b

Then check the split:

ollama ps

On 16 GB you will see something like a CPU/GPU split rather than “100% GPU”. To get everything onto the GPU, run Unsloth’s 3-bit GGUF straight from Hugging Face instead:

ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:UD-Q3_K_XL

Set a sensible context for 16 GB (Ollama’s default is smaller than the model’s maximum):

# per session
/set parameter num_ctx 16384

# or for the server (Windows: set it as a user environment variable, then restart Ollama)
OLLAMA_CONTEXT_LENGTH=16384
OLLAMA_FLASH_ATTENTION=1
OLLAMA_KV_CACHE_TYPE=q8_0

OLLAMA_KV_CACHE_TYPE=q8_0 halves the KV cache memory with very little quality cost and needs flash attention enabled.

Thinking is on by default in Ollama’s build and can be switched off per request; in the CLI use /set nothink, and in the API send "think": false. Ollama’s page also mentions a reasoning_effort option to tune how long it thinks.

More commands are in my Ollama commands cheat sheet, and if Ollama ignores your GPU, see how to fix Ollama not using the GPU.

Run It in LM Studio on 16 GB

1. Update LM Studio and its runtimes (Settings > Runtime) so the llama.cpp engine knows the qwen35 architecture.
2. In Discover, search Qwen3.8-27B, open the Unsloth GGUF and download UD-Q3_K_XL.
3. When loading, set GPU Offload to the maximum (all layers), Context Length to 16384 to start, turn on Flash Attention, and set the K and V cache quantization to Q8_0.
4. Watch the VRAM estimate in the load dialog. If it shows the model spilling over, lower the context first.

For vision, download the mmproj file from the same repo; LM Studio pairs it with the model automatically when both are in the same folder. Not sure which app to use? My Ollama vs LM Studio vs llama.cpp comparison helps.

Run It With llama.cpp (Best Control)

llama.cpp gives you the most control over memory. Use a current build: the GGUF uses the new qwen35 architecture and older builds refuse to load it. Download the 3-bit file and the vision projector:

pip install -U huggingface_hub
hf download unsloth/Qwen3.8-27B-GGUF --local-dir qwen38-27b --include "*UD-Q3_K_XL*" --include "*mmproj-BF16*"

Text-only server on a 16 GB card, everything on the GPU, quantized KV cache:

llama-server -m qwen38-27b/Qwen3.8-27B-UD-Q3_K_XL.gguf -ngl 99 -c 32768 -fa on --cache-type-k q8_0 --cache-type-v q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --port 8080

Add vision by loading the projector (and lower the context a little to make room for it):

llama-server -m qwen38-27b/Qwen3.8-27B-UD-Q3_K_XL.gguf --mmproj qwen38-27b/mmproj-BF16.gguf -ngl 99 -c 16384 -fa on --cache-type-k q8_0 --cache-type-v q8_0 --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0

For Q4_K_M on 16 GB, replace -ngl 99 with a number of layers that fits (start around -ngl 56 of 64 and raise it until you run out of VRAM).

The server is OpenAI-compatible at http://localhost:8080/v1, so Cline, Continue, OpenCode and other tools can use it. Recent llama.cpp versions also accept the unified llama serve form with the same options.

Thinking Mode, Sampling and Vision Settings

Thinking produces better code and maths, but on a 16 GB card it also eats context quickly, because the reasoning is stored in the KV cache. For chat, short edits and summaries, switch it off.

For non-thinking use, check the model card for Qwen’s recommended non-thinking sampling values, which are usually a little lower than the thinking ones.

Two GPUs, 24 GB and RTX 50 Options

• 24 GB (RTX 3090, 4090): UD-Q4_K_XL (17.9 GB) fully on the GPU with 32K-64K context, or UD-Q5_K_XL with less context.
• 32 GB (RTX 5090): UD-Q6_K_XL (25.9 GB) with a comfortable context.
• Two 16 GB GPUs (my setup, 2x RTX 5080): llama.cpp splits layers across both cards automatically. UD-Q5_K_XL (20.2 GB) or UD-Q6_K_XL (25.9 GB) fit with room for a long context. Add --tensor-split 1,1 for an even split. Ollama also spreads a model over two GPUs when it does not fit on one.
• RTX 50 with vLLM or SGLang: there are NVFP4 builds that use Blackwell’s 4-bit tensor cores and an FP8 KV cache. They do not run in Ollama or LM Studio. See my RTX 50 setup guide for what NVFP4 is.

Faster Alternatives for 16 GB

If Qwen3.8-27B at 3-bit is too slow or too compressed for you, these run more comfortably on 16 GB:

• Gemma 4 26B A4B: a mixture-of-experts with only 3.8B active parameters, so it generates much faster, and its 4-bit QAT GGUF is 14.2 GB.
• Qwen3.6 35B-A3B (qwen3.6:35b): Qwen’s previous MoE, also fast thanks to about 3B active parameters, and good at code.
• MiMo-V2.6 9B: much smaller, fits fully with a big context, and strong at coding for its size.
• Small Qwen models such as qwen3.5:2b, qwen3-vl:8b and qwen3-coder:30b for laptops, quick vision tasks and a fast MoE coder.

Common Problems and Fixes

Frequently Asked Questions

Can Qwen3.8-27B run on a 16 GB GPU?

Yes. Unsloth’s UD-Q3_K_XL GGUF is 13.4 GB and runs fully on a 16 GB card with a 16K-32K context. Ollama’s default 18 GB build also runs, but part of it spills to the CPU and it is slower.

What is the best quant of Qwen3.8-27B for 16 GB VRAM?

UD-Q3_K_XL (13.4 GB) for speed with everything on the GPU, or Q4_K_M (17.1 GB) with a small CPU offload if you prefer quality over speed.

How much VRAM does Qwen3.8-27B need?

About 9 GB at UD-IQ2_XXS, 13.4 GB at UD-Q3_K_XL, 17-18 GB at 4-bit, 20-26 GB at 5-6 bit and 29-31.5 GB at 8-bit, plus room for context and about 0.93 GB for the vision projector.

Is Qwen3.8-27B free for commercial use?

Yes. Qwen3.8-27B is released under the Apache 2.0 licence.

How do I turn off thinking in Qwen3.8-27B?

Thinking is on by default. In Ollama use /set nothink in the CLI or send think: false in the API. In llama.cpp and LM Studio, use the chat template option for disabling thinking.

Does Qwen3.8-27B support images and video?

Yes, it is a native vision-language model. Load the mmproj-BF16.gguf projector in llama.cpp or LM Studio, or use Ollama’s qwen3.8:27b, which accepts images.

Is Qwen3.8-27B better than Gemma 4?

On Alibaba’s and Google’s own benchmarks, Qwen3.8-27B leads on coding and agent tasks, while Gemma 4 31B is very strong at maths and reasoning. On 16 GB, Gemma 4 26B A4B is faster because it is a mixture-of-experts.

Final Thoughts

A 16 GB card is enough for Qwen3.8-27B if you choose the file carefully: UD-Q3_K_XL fully on the GPU for daily use, Q4_K_M with a small offload when you want maximum quality, and a quantized KV cache to make room for context. With 24 GB or two 16 GB cards you can move up to 4-6 bit and get close to full quality. For more choices by VRAM, see my best Ollama models list, and if you want to clean up old downloads afterwards, here is how to delete Ollama models.

Sources

• DataNorth: Alibaba releases Qwen3.8-27B (architecture, licence, benchmarks)
• unsloth/Qwen3.8-27B-GGUF on Hugging Face
• OrcaRouter: Qwen3.8-27B Unsloth GGUF sizes and settings
• Ollama library: qwen3.8
• Qwen/Qwen3.8-27B model card