Qwen3.8-27B is one of the best models you can run on a single 24 GB card. Its weak spot is speed: with plain decoding it writes about 28 tokens per second on an RTX 3090.
DSpine, a small open “draft” model from Tencent, can push that to around 90 to 128 tokens per second on code and mixed chat, according to community tests.
The catch is that it is not a one-click install yet. This guide explains what DSpine is, which tools run it today (SGLang and a llama.cpp fork), the exact commands, the context and VRAM trade-off on 24 GB cards, and when it is not worth it.
- What: a five-layer speculative decoding drafter for Qwen3.8-27B (tencent/Qwen3.8-27B-DSpine-b16), Apache 2.0. It guesses 16 tokens at once and the big model checks them in one pass.
- Speed: RTX 3090 community test went from 28.2 tok/s to 90.1 (mixed), 128.7 (code) and 41.6 (prose). Tencent reports 3.84x throughput in its own agent replay at 8 concurrent requests.
- Runs on: SGLang 0.5.22 (official, CUDA, BF16) or a community llama.cpp
dspinebranch with a 1.26 GB Q4_K_M GGUF. Not in upstream llama.cpp, Ollama, LM Studio or vLLM yet. - Cost: context. On 24 GB, max context drops from about 176K to 65K tokens with 15 drafted tokens.
- Skip it if: you mostly write prose, use a Mac, or run AMD on Vulkan without tuning.
New to this model? Start with my guide to running Qwen3.8-27B on a 16 GB GPU, then come back here to make it faster.
What DSpine Is
DSpine is a speculative decoding drafter. A small, fast model proposes the next few tokens, and the big model (here Qwen3.8-27B) checks all of them in one forward pass. Every token the big model agrees with is kept for free.
If it disagrees, it keeps its own token and the draft restarts from there, so the final text comes from the big model.
What makes DSpine different from older drafters is that it drafts a whole block of 16 tokens in parallel, instead of one after another.
Per Tencent’s README, at every draft layer each position gets the predicted feature of the token before it, which keeps the guesses consistent with each other. The method is described in the paper “Draft in Parallel, Condition Through Depth” (arXiv 2609.36173).
| Detail | Value |
|---|---|
| Developer | Tencent (Tencent/DSpine) |
| Target model | Qwen3.8-27B (a separate drafter exists for Qwen3-8B) |
| Draft layers | 5 |
| Block size | 16 tokens |
| Sliding window | 8,192 tokens |
| Licence | Apache 2.0 |
| Released | Paper and Qwen3-8B drafter in October 2026; Qwen3.8-27B support in November 2026 |
Important: the drafter is trained for one target. It works with Qwen3.8-27B and its quantised GGUF versions, but not with other models, and a fine-tune of Qwen3.8-27B would need a retrained drafter to get the same acceptance.
How Much Faster Is It?
The speed-up depends on how often the big model accepts the draft. Code and maths are predictable, so acceptance is high. Creative prose is not, so gains shrink.
| Setup | Without DSpine | With DSpine | Source |
|---|---|---|---|
| RTX 3090, Q4_K_M, code prompts | 28.2 tok/s | 128.7 tok/s (7.6 tokens accepted per step) | Community test (llama.cpp branch) |
| RTX 3090, Q4_K_M, mixed prompts | 28.2 tok/s | 90.1 tok/s | Community test |
| RTX 3090, Q4_K_M, prose | 28.2 tok/s | 41.6 tok/s (2.5 accepted per step) | Community test |
| SGLang BF16, 8 requests, WorkBuddy Office replay | 210 tok/s | 806 tok/s (3.84x); DFlash2 665 | Tencent |
| SGLang BF16, 8 requests, SWE-bench Pro replay | 242 tok/s | 713 tok/s; DFlash2 658 | Tencent |
| DGX Spark, Q8_0 target | – | 29.5 tok/s vs DFlash2 29.6 | llama.cpp PR test |
| RX 7800 XT (Vulkan), 15 drafted tokens | 30.9 tok/s | 19.1 tok/s (slower); 47.4 with 2 drafted | Community test |
Tencent’s own acceptance numbers for Qwen3.8-27B (tokens accepted per pass, DSpine vs DFlash2) are GSM8K 7.20 vs 5.69, MATH-500 6.49 vs 5.49, HumanEval 5.71 vs 4.87, MBPP 6.14 vs 5.04 and MT-Bench 4.48 vs 4.11. These are vendor figures.
One reviewer points out that DSpine drafts 16 tokens while DFlash2 drafts 8, which explains part of the gap.
The SGLang figures are total throughput across 8 users, not the speed one person sees. For a single chat on your own PC, the RTX 3090 numbers are the better guide.
Which Apps Support It
| App | DSpine for Qwen3.8-27B? | Notes |
|---|---|---|
| SGLang 0.5.22 | Yes (official) | CUDA, BF16, FlashAttention 3. Installed by Tencent’s repo |
| llama.cpp (upstream) | No | Draft PR #31941 is open. Stock builds refuse the GGUF files |
llama.cpp dspine branch |
Yes (community) | Ajay9o9/llama.cpp, tested on CUDA |
| vLLM | No | No support reported as of 9 Nov |
| Ollama, LM Studio | No | Both build on llama.cpp, so wait for the upstream merge |
| Mac (MLX) | Qwen3-8B only | Tencent’s dspine_mlx covers full-attention Qwen3 targets, not the hybrid 27B |
If you only use Ollama or LM Studio, bookmark this page. Once the llama.cpp PR merges, both apps usually pick it up in a later release. Our llama.cpp 1.0 guide explains how releases and tags work now.
Files and VRAM
You need two files: the normal Qwen3.8-27B GGUF (the target) and the DSpine drafter GGUF from aj9o9/Qwen3.8-27B-DSpine-b16-GGUF.
| File | Size | Use |
|---|---|---|
Qwen3.8-27B-Q4_K_M.gguf (target, e.g. Unsloth) |
about 16.5 to 17.1 GB | The main model |
Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf |
1.26 GB | Recommended drafter |
Qwen3.8-27B-DSpine-b16-Q8_0.gguf |
2.20 GB | Same acceptance, slower, more VRAM |
Qwen3.8-27B-DSpine-b16-BF16.gguf |
4.06 GB | Only for making your own quants |
Download them with the Hugging Face CLI:
hf download aj9o9/Qwen3.8-27B-DSpine-b16-GGUF Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf --local-dir models
Qwen3.8-27B is a hybrid model with recurrent layers, so speculative decoding also keeps copies of that state. The community test measured about 157 MB per copy, or 2.35 GB for 15 drafted tokens. That is why context shrinks so much:
| 24 GB card, Q4_K_M target, q8_0 KV cache | Max context |
|---|---|
| No drafter | 176,128 tokens |
| DSpine Q4_K_M, 7 drafted tokens | 98,304 tokens |
| DSpine Q4_K_M, 15 drafted tokens | 65,536 tokens |
| DSpine Q8_0, 15 drafted tokens | 40,960 tokens |
On a 16 GB card the Q4_K_M pair (about 17.7 GB of weights) does not fit fully in VRAM, so DSpine is really a 24 GB-and-up trick for now.
Method 1: llama.cpp DSpine Branch (Single GPU)
This is the easiest way to try DSpine on a home PC with an NVIDIA card. You build a fork of llama.cpp that adds DSpine support. You need Git, CMake and the CUDA Toolkit installed.
- Clone and build the branch:
git clone -b dspine https://github.com/Ajay9o9/llama.cpp.git cd llama.cpp cmake -B build -DGGML_CUDA=ON cmake --build build --config Release -j - Start the server with both models:
./build/bin/llama-server \ -m Qwen3.8-27B-Q4_K_M.gguf \ -md Qwen3.8-27B-DSpine-b16-Q4_K_M.gguf \ --spec-type draft-dflash \ --spec-draft-n-max 15 \ -ngl 999 --spec-draft-ngl 999 \ -fa on --jinja -c 32768 -np 1 \ --cache-type-k q8_0 --cache-type-v q8_0 \ --spec-draft-type-k q8_0 --spec-draft-type-v q8_0On Windows the binary is
build\bin\Release\llama-server.exe. - Open
http://127.0.0.1:8080and chat, or point any OpenAI-compatible app athttp://127.0.0.1:8080/v1.
What the key flags do:
-md: the draft model file.--spec-type draft-dflash: the block-drafting mode that DSpine uses in this branch.--spec-draft-n-max 15: how many tokens to draft. The default is 3, which wastes most of the 16-token block. 15 was best on NVIDIA in the branch’s tests.-c 32768: set the context yourself. Auto-fit does not leave room for the drafter and the state copies.
This branch is a community project, not an official llama.cpp release. Keep your normal llama.cpp build for everyday use and run this one side by side.
Method 2: SGLang (Official)
Tencent’s official path is SGLang on Linux with CUDA and full BF16 weights. It suits a big GPU (the BF16 27B model alone is far more than 24 GB) or a server that handles several users.
- Install (Python 3.11+ and CUDA required; this installs SGLang 0.5.22):
git clone https://github.com/tencent/dspine.git cd dspine python -m pip install -e . - Serve (core flags from the README; the full command adds Mamba cache and CUDA graph options):
sglang serve \ --model-path /path/to/Qwen3.8-27B \ --speculative-algorithm DSPINE \ --speculative-draft-model-path tencent/Qwen3.8-27B-DSpine-b16 \ --speculative-num-draft-tokens 16 \ --tp-size 1 --dtype bfloat16 \ --attention-backend fa3 --speculative-draft-attention-backend fa3 \ --linear-attn-backend triton --enable-linear-replayssm-spec \ --mamba-ssm-dtype float32 \ --mem-fraction-static 0.80 --context-length 131072 \ --max-running-requests 8 \ --reasoning-parser qwen3 --tool-call-parser qwen3_coder \ --trust-remote-code
Copy the full command from the DSpine README before you run it, because it also sets the Mamba cache strategy, CUDA graph batch sizes and the sampling backend. The README describes this setup as a single GPU with up to eight concurrent requests.
Tuning: Draft Length and Context
- NVIDIA (CUDA): start at
--spec-draft-n-max 15. - Need more context or 2 slots on 24 GB: drop to 7. With
-np 2, use 7. - AMD on Vulkan: try 2 or 3. At 15 the RX 7800 XT got slower than no drafter at all.
- Mostly prose: try 3, or use the target’s own MTP head or DFlash2 instead.
- Out of memory: lower
-cfirst, then the draft length. Use the Q4_K_M drafter, not Q8_0.
A simple rule from the community test: each verify pass must return more than about 1.7 tokens to beat plain decoding on that GPU. If the server log shows an average acceptance under 2 at 15 drafted tokens, the drafter is costing you context without paying it back.
When DSpine Is Not Worth It
- Story writing and long prose. Gains fall to around 1.5x and you lose a lot of context.
- Very long documents. If you need 100K+ tokens of context on 24 GB, plain decoding keeps it.
- Mac users. There is no published Metal result for the 27B drafter yet.
- Ollama or LM Studio only. Wait for upstream support instead of building a fork.
- Exact reproducibility. Even at temperature 0, output can differ slightly from runs without a drafter, mostly in prose and image descriptions, because batched checks can flip near-tied tokens.
For coding agents, tool calls and maths on a 24 GB NVIDIA card, it is one of the biggest free speed-ups available for this model right now.
Troubleshooting
| Problem | Fix |
|---|---|
| Stock llama.cpp refuses to load the drafter | Expected. Build the dspine branch; upstream support is still a draft PR. |
Log shows failed to allocate ... using host copies |
Context is too big. Lower -c, or drop --spec-draft-n-max to 7. |
| No speed-up at all | Check that --spec-draft-n-max is set (default 3 is too low on CUDA) and that both models are on the GPU (-ngl 999 --spec-draft-ngl 999). |
| Slower than without DSpine (AMD/Vulkan) | Lower draft length to 2 or 3. |
| Fast on code, slow on chat | Normal: prose has low acceptance. Use a smaller draft length for chat. |
| Out of memory with two slots | Use -np 2 with draft length 7, or one slot. |
| Draft used with a Qwen3.8-27B fine-tune gives little gain | The drafter is trained on the base model. Fine-tunes need a retrained drafter. |
| SGLang install fails | You need Python 3.11+, CUDA and an NVIDIA GPU supported by FlashAttention 3. The repo pins SGLang 0.5.22. |
FAQ
Is DSpine free?
Yes. The code and drafter weights are Apache 2.0, and Qwen3.8-27B itself is Apache 2.0.
Does DSpine change the answers?
Mostly no, because the big model checks every token. Small differences can appear at temperature 0 where two tokens are almost tied, mainly in prose.
Can I use it in Ollama?
Not yet. Ollama uses llama.cpp, and DSpine is not in upstream llama.cpp. Watch PR #31941.
Will it work on a 16 GB GPU?
Not fully on the GPU. The Q4_K_M target plus drafter is about 17.7 GB of weights before any context.
What about Qwen3-8B?
Tencent also released tencent/Qwen3-8B-DSpine-b16, and that one has an official MLX path for Apple Silicon. Tencent reports 15.37 to 38.52 tok/s on an M4 Pro with Qwen3-8B in BF16.
Is this the same as DFlash2 or MTP?
They are all speculative decoding methods. DFlash2 is an earlier block drafter (8 tokens); MTP uses extra prediction heads built into some models. DSpine had higher acceptance than DFlash2 in Tencent’s tests.
Sources
- Hugging Face: tencent/Qwen3.8-27B-DSpine-b16 (model card, vendor numbers)
- GitHub: Tencent/DSpine README (install, SGLang command, Qwen3-8B tables)
- Hugging Face: aj9o9/Qwen3.8-27B-DSpine-b16-GGUF (GGUF files, llama.cpp branch commands)
- Rohit Ghumare: DSpine on a 24 GB card (RTX 3090, context, AMD and Mac measurements)
- ModelFit: DSpine on a Mac (file sizes, Mac status, PR #31941)




