Run Qwen3.8-27B Locally in GitHub Copilot With Foundry Local (2026): 24 GB GPU Setup Guide

Microsoft just showed how to run a proper agentic coding model on your own PC and plug it straight into GitHub Copilot. The model is Alibaba’s Qwen3.8-27B, served locally by Foundry Local, with no Azure account and no per-token bill.

The guide landed on Microsoft’s Foundry blog on 5 November 2026, together with Foundry Local CLI 0.13.0. The catch is hardware: you need an NVIDIA card with 24 GB of VRAM or more.

Below you get everything in one place: specs, benchmarks, the exact commands, how to connect the Copilot app and VS Code, which GPUs work, how it compares with Ollama and LM Studio, fixes for common errors, and an honest look at the limits.

What Microsoft Announced

The post is called “Bring Agentic Coding On-Device with Foundry Local”. It was written by Baiju Meswani, Emma Ning and Tianlei Wu from the Microsoft Foundry team.

Foundry Local is Microsoft’s free runtime for running AI models on your own machine. It is built on ONNX Runtime (onnxruntime-genai) and exposes an OpenAI-style API on localhost, so any tool that speaks that API can use it.

What is new is a coding-grade model tuned for it. Microsoft calls Qwen3.8-27B its “primary optimization target” and lists the work it did:

• INT4 weights plus KV-cache quantization, so the model fits in about 18 GB.
• Speculative decoding (named “DFlash2” in the post) to speed up generation.
• CUDA graph fusions and kernel tuning, including for the SM121 architecture used by DGX Spark.
• Prefix caching and continuous batching, which help when an agent sends the same long context again and again.
• Embedding memory sharing and CPU offloading to stretch VRAM.

The recipe is public in Microsoft’s olive-recipes GitHub repo under Qwen-Qwen3.8-27B/cuda, so you can see how the model was converted.

Note: Qwen3.8-27B itself is not new. Alibaba released it in August 2026. The news is that Microsoft now ships an optimised build and an official path into GitHub Copilot.

Qwen3.8-27B Specs and Benchmarks

Here are the coding and agent benchmarks from the Qwen model card. These are vendor-reported and were not run inside Copilot.

So a 27B model that fits on one gaming card lands within a few points of a top cloud model on terminal tasks. Windows Forum notes that some of these runs used a Claude Code style harness, so results inside Copilot may differ.

How fast is it?

Microsoft compared Foundry Local with llama.cpp on an RTX 5090 (32 GB) and a DGX Spark (128 GB unified memory), using prompts built from real GitHub issues. It says Foundry Local generated faster at every prompt length tested, and that the lead grew beyond 131K tokens.

The post shows the numbers only in charts. One developer write-up reads them as roughly 1.5 to 1.9 times llama.cpp on the 5090. Microsoft has not published the llama.cpp version or settings, so treat this as a vendor claim until someone replicates it.

Which GPU Do You Need?

Microsoft’s own wording is clear: an NVIDIA GPU with at least 24 GB of VRAM. Here is what that means in practice.

Tip: On a 16 GB card you can still run this model with a smaller GGUF quant in Ollama or LM Studio. Our guide to running Qwen3.8-27B on a 16 GB GPU has the exact files and settings.

Other requirements:

• Windows 11 on x64 (Microsoft’s steps use WinGet). ARM64 builds need CUDA 13.
• CUDA 12 and cuDNN 9 on x64, with their DLL folders on your PATH. Foundry Local does not install them.
• Foundry Local CLI 0.13.0 or newer (public preview, released 2 November 2026).
• Disk: about 18 GB for the model, plus room for CUDA.

Not sure what your card has? See how to check GPU VRAM and usage. Shopping for one? Our best GPU for local LLMs list and the RTX 5090 vs RTX 5080 for AI comparison cover prices.

Step by Step: Install Foundry Local and Load the Model

Plan for about 30 to 60 minutes, most of it downloading CUDA and the model. Open a new PowerShell window for each step that changes PATH.

1. Check your GPU and driver. Run nvidia-smi. You should see your card name and at least 24 GB under memory. If the command is missing, install the latest NVIDIA driver and reboot.
2. Install CUDA 12 and cuDNN 9. Use NVIDIA’s installers, then make sure both DLL folders are on your system PATH. Check with nvcc --version and where.exe cudnn64_*.dll.
3. Install Foundry Local. Run winget install Microsoft.FoundryLocal. If you already have it, run winget upgrade Microsoft.FoundryLocal.
4. Confirm the version. Run foundry --version. It must show 0.13.0 or higher.
5. Restart the service if needed. If Foundry Local was already running before you changed PATH, run foundry server restart from a fresh terminal.
6. Find the model ID. Run foundry model list. Microsoft uses the alias qwen3.8-27b; one third-party guide shows qwen3-8-27b, so use whatever your catalog prints.
7. Download and load it. Run foundry model load qwen3.8-27b. The first run downloads about 18 GB. Watch nvidia-smi: VRAM use should jump. If it does not, the runtime fell back to CPU (see Troubleshooting).
8. Get your endpoint. Run foundry server status. Copy the address it prints, which looks like http://127.0.0.1:PORT/v1. The port is picked on your machine, so do not copy one from a guide.

Warning: Close Ollama, LM Studio, ComfyUI and games before loading. Another app holding 6 to 8 GB of VRAM is the most common reason the load fails on a 24 GB card.

Test the Local Endpoint

Before touching Copilot, make sure the server answers. Replace PORT with your number from foundry server status.

List the loaded models:

curl.exe http://127.0.0.1:PORT/v1/models

Send a quick chat request from PowerShell:

$body = @{
  model = "qwen3.8-27b"
  messages = @(@{ role = "user"; content = "Write a Python function that reverses a string. Code only." })
  max_tokens = 800
} | ConvertTo-Json -Depth 5

Invoke-RestMethod -Uri "http://127.0.0.1:PORT/v1/chat/completions" -Method Post -ContentType "application/json" -Body $body

Thinking mode spends tokens before it answers, so keep max_tokens generous. If the reply is empty or cut off, raise it.

The Qwen team recommends these sampling settings if your client lets you set them:

Connect GitHub Copilot (App and VS Code)

GitHub calls this local BYOK. Per GitHub’s docs it works in VS Code, JetBrains IDEs, Xcode, Copilot CLI, the GitHub Copilot app and the Copilot SDK, and it does not depend on the Copilot API.

In the GitHub Copilot app (Microsoft’s documented path)

1. Open Settings in the GitHub Copilot app and go to Model providers.
2. Choose Add provider and select Foundry Local.
3. Paste your endpoint from foundry server status into Base URL.
4. Open Browse Catalog, search for Qwen3.8-27B and add it.
5. Pick the model. Set the provider to Foundry Local and choose Qwen3.8-27B in the model picker.

In VS Code

Microsoft’s post only documents the Copilot app. In VS Code the flow is similar, but menu names change between Copilot Chat builds:

1. Open Copilot Chat and click the model picker, then Manage Models.
2. Add a provider. Pick Foundry Local if your build lists it, or an OpenAI-compatible provider if not.
3. Enter the endpoint including /v1. If a key is required, any placeholder such as local works, because the local server does not check it.
4. Select Qwen3.8-27B in agent mode and give it a real task on an open file.

Watch nvidia-smi -l 1 while it works. GPU use should spike, which proves the request is running locally.

Note: On Copilot Business and Enterprise, an organisation policy can switch off local BYOK in IDEs. If you see no provider option, ask your admin. Local BYOK is described as usable without a Copilot subscription.

If you want other agent front ends for the same local model, our Cline, Continue and Roo Code in VS Code guide works with any OpenAI-compatible endpoint, and GitHub Copilot in Xcode covers the Mac side of Copilot.

Foundry Local vs Ollama vs LM Studio vs llama.cpp

Pick Foundry Local if you have a 24 GB+ NVIDIA card on Windows and work with big repos, where its long-context speed should matter most. Pick Ollama or LM Studio for smaller cards, Macs or AMD. Our Ollama vs LM Studio vs llama.cpp guide goes deeper, and best Ollama models by VRAM lists alternatives.

Compared with cloud agents such as Claude Code or Cursor, you trade some reasoning power for privacy and zero running cost. Our Cursor vs Windsurf vs Claude Code vs Cline comparison covers that side.

Privacy: What Stays on Your PC

• Stays local: the prompts and code sent to the model, and the model’s replies. GitHub’s BYOK docs say requests go from the client straight to your chosen provider.
• May still use the network: Copilot sign-in, extensions, some tool calls, and features such as inline completions that may still use GitHub’s own models.
• Microsoft side: the model catalog and Foundry Local telemetry are run by Microsoft.

A simple test: disconnect from the internet and check that agent chat with Qwen3.8-27B still answers. If you need a strict air gap, confirm every Copilot feature you use with your security team.

Troubleshooting

Limits, Licence and Open Questions

• Licence: Qwen3.8-27B is Apache 2.0, so commercial use is fine. Foundry Local is a free Microsoft preview.
• NVIDIA only for now: Microsoft says more devices, including Macs and AMD GPUs, are on the roadmap, with no date.
• Weaker than the best cloud models: Microsoft itself says local models have “limited reasoning capabilities” and suggests a hybrid setup, local for routine work and cloud for hard problems.
• Multi-GPU: the post does not say whether Foundry Local can split this model across two cards, so do not count on two 16 GB GPUs working.
• Unverified speed: no tokens-per-second figures in text and no llama.cpp settings, so wait for independent tests.
• Preview software: menus, model IDs and ports may change between builds.

My Take

This is the first time a big vendor has given ordinary developers an official, tuned path from “local model” to “agent inside Copilot”. That matters more than the benchmark numbers.

A 73 on Terminal Bench 2.1 from a model that fits on an RTX 4090 is genuinely useful for refactors, tests and repo chores. For gnarly architecture work I would still reach for a cloud model, exactly as Microsoft suggests.

The 24 GB floor is the real limit. If you own a 3090, 4090 or 5090, try it today. If you are on a 16 GB card, stay with a GGUF build in Ollama or LM Studio until Microsoft adds smaller variants.

FAQ

Is Foundry Local free?

Yes. Foundry Local is a free Microsoft runtime in public preview, it needs no Azure subscription, and there is no per-token charge.

Do I need a paid GitHub Copilot plan?

GitHub’s docs describe local BYOK as usable without a Copilot subscription. On Business or Enterprise plans, your admin can disable it with a policy.

Can I run it on a 16 GB GPU like the RTX 5080?

Not with Microsoft’s build, which needs 24 GB. You can run smaller GGUF quants of Qwen3.8-27B on 16 GB through Ollama, LM Studio or llama.cpp.

Does it work on a Mac or an AMD GPU?

Not yet. Microsoft lists Macs and AMD GPUs as planned, without a date.

Is my code sent to GitHub or Microsoft?

Prompts to the local model stay on your machine. Sign-in, extensions and some other Copilot features can still use the network.

What context length can I use?

Up to about 64K tokens on a 24 GB card and the model’s full 262K on cards with 32 GB or more, per Microsoft.

How good is Qwen3.8-27B at coding?

Qwen reports 73.0 on Terminal Bench 2.1 and 61.7 on SWE-bench Pro, a few points behind Claude Opus 4.6 on terminal tasks. These are vendor numbers, not Copilot tests.

Sources

• Microsoft Foundry Blog: “Bring Agentic Coding On-Device with Foundry Local” (5 Nov 2026), devblogs.microsoft.com/foundry/foundry-local-agentic-coding-qwen3-8-27b/
• Qwen3.8-27B model card, huggingface.co/Qwen/Qwen3.8-27B
• GitHub Docs: Bring your own key (BYOK), docs.github.com/en/copilot/concepts/models/bring-your-own-key
• Foundry Local releases, github.com/microsoft/foundry-local/releases
• Windows Forum: “Run Qwen3.8-27B Locally with GitHub Copilot and Foundry Local on Windows” (CUDA/cuDNN versions, troubleshooting)
• Tech Insider: step-by-step Foundry Local + Copilot setup (model ID, PowerShell tests)
• AI Weekly alert, 6 Nov 2026: “Microsoft Foundry Local runs Qwen3.8-27B on-device for Copilot”