Skip to content
,

How to Set Up WSL2 for Local AI on Windows: GPU, CUDA, Ollama and Docker (2026)

9–14 minutes
Thrilled woman in a green jacket in a neon-lit tech lab holding a white card reading WSL2, hook text: Linux AI on Windows

Lots of local AI tools are built for Linux first. Install scripts, CUDA builds, Python packages and GitHub projects often assume Ubuntu, and Windows users end up fighting missing wheels and path errors.

WSL2 (Windows Subsystem for Linux) fixes that. It runs a real Ubuntu system inside Windows, and with an NVIDIA card it can use your GPU through CUDA at near-native speed.

This guide takes you from a clean Windows PC to a working local AI setup in WSL2: install, GPU check, CUDA, Python, Ollama, Docker, memory limits and fixes for the common errors. Every command comes from Microsoft’s WSL docs, NVIDIA’s CUDA on WSL guide and Ollama’s Linux docs, checked in November 2026.

TL;DR

  • Install: open PowerShell as admin, run wsl --install, restart, then create a Linux user.
  • GPU: install only the normal NVIDIA Windows driver. Never install a Linux NVIDIA driver inside WSL.
  • Check: run nvidia-smi inside Ubuntu. If it lists your card, the GPU is ready.
  • CUDA toolkit: only needed to compile code. Use NVIDIA’s WSL-Ubuntu installer or the cuda-toolkit package, never cuda or cuda-drivers.
  • Ollama: curl -fsSL https://ollama.com/install.sh | sh inside Ubuntu.
  • Speed: keep models and projects in your Linux home folder, not on /mnt/c.
  • RAM: WSL2 gets 50% of your RAM by default. Raise it in %UserProfile%\.wslconfig or WSL Settings.

Why Use WSL2 for Local AI?

Native Windows apps like LM Studio and the Ollama Windows app are fine for simple chat. WSL2 starts to matter when you go beyond that.

  • Linux-first projects: many GitHub AI repos only document Linux setup, and some Python packages ship Linux CUDA wheels first.
  • Same commands as tutorials: most guides use bash, apt and Linux paths, so you can copy them as written.
  • GPU access: WSL2 passes your NVIDIA GPU through to Linux, so CUDA apps run on the card.
  • No dual boot: you keep Windows for games and apps and open Ubuntu in a terminal tab.
  • Clean sandbox: you can break and reinstall a Linux distro without touching Windows.
Option Best for Trade-off
Native Windows apps Simple chat, one-click tools Some projects lack Windows support
WSL2 (Ubuntu) Python projects, servers, Linux-only tools Some limits on pinned and managed GPU memory
Dual boot Linux Maximum performance, heavy training Restart to switch, more setup

Requirements

Item Requirement
Windows Windows 11, or Windows 10 version 2004+ (build 19041+) for wsl --install. GPU compute needs Windows 11 or Windows 10 21H2+.
Virtualization Enabled in BIOS/UEFI (Intel VT-x or AMD-V/SVM)
NVIDIA GPU Pascal (GTX 10-series) or newer recommended; Maxwell is not officially supported
Driver Latest NVIDIA Windows display driver (GeForce or RTX/Quadro)
Disk At least 50 GB free; models add 4 to 50+ GB each
RAM 16 GB minimum, 32 GB+ recommended for bigger models

Step 1: Install WSL2 and Ubuntu

  1. Open PowerShell as administrator: right-click Start, choose Terminal (Admin) or Windows PowerShell (Admin).
  2. Run the installer: wsl --install. This turns on the features WSL needs and installs Ubuntu.
  3. Restart your PC when it asks.
  4. Open Ubuntu from the Start menu. The first launch unpacks files, then asks you to create a Linux username and password. This password is for sudo inside Linux only.
  5. Update WSL itself: in PowerShell run wsl --update to get the latest kernel.
  6. Confirm the version: run wsl -l -v. The VERSION column must say 2. If it says 1, run wsl --set-version Ubuntu 2.
  7. Update Ubuntu packages: inside Ubuntu run sudo apt update && sudo apt upgrade -y.

If wsl --install only prints help text, WSL is already present. Run wsl --list --online to see distros, then wsl --install -d Ubuntu. If the install hangs at 0.0%, Microsoft’s fix is wsl --install --web-download -d Ubuntu.

Step 2: Give WSL2 Your NVIDIA GPU

This is the step most people get wrong. NVIDIA is clear that the Windows driver is the only driver you need. It is exposed inside WSL2 as a stub library, libcuda.so, so Linux apps can talk to the card.

  1. Install or update the NVIDIA driver on Windows from nvidia.com or the NVIDIA app. A normal Game Ready or Studio driver works.
  2. Do not install any Linux NVIDIA driver inside Ubuntu. Skip apt install nvidia-driver-* and the .run driver installers.
  3. Restart WSL after a driver update: in PowerShell run wsl --shutdown, then open Ubuntu again.
  4. Test it: inside Ubuntu run nvidia-smi. You should see your GPU name, driver version and VRAM.

If nvidia-smi is “not found”, try /usr/lib/wsl/lib/nvidia-smi. NVIDIA notes it lives there in WSL, and you can add /usr/lib/wsl/lib to your PATH. Note that some GPU stats, like utilisation and the process list, are not supported inside WSL, so blank fields there are normal.

Step 3: Install the CUDA Toolkit (Optional)

You do not need the full CUDA toolkit to run Ollama or to use PyTorch from pip, because those ship their own CUDA runtime. You need it only if you compile CUDA code yourself, for example building llama.cpp from source or some custom kernels.

  1. Remove NVIDIA’s old signing key: sudo apt-key del 7fa2af80
  2. Open the CUDA Toolkit download page on developer.nvidia.com and choose Linux, x86_64, WSL-Ubuntu, then a deb installer.
  3. Run the commands it shows. The WSL-Ubuntu package leaves out the Linux GPU driver, so it cannot overwrite the WSL driver.
  4. If you use apt meta-packages, install only cuda-toolkit-12-x (or the matching toolkit-only package for your version).
  5. Check it: nvcc --version. If it is not found, add /usr/local/cuda/bin to your PATH in ~/.bashrc.

Warning: do not install the cuda, cuda-12-x or cuda-drivers meta-packages. NVIDIA says these try to install a Linux driver, which breaks GPU access in WSL.

Step 4: Set Up Python for AI

  1. Install basics: sudo apt install -y python3-venv python3-pip git build-essential
  2. Make a project folder in Linux: mkdir -p ~/ai && cd ~/ai
  3. Create a virtual environment: python3 -m venv .venv && source .venv/bin/activate
  4. Install PyTorch with CUDA: copy the Linux + pip + CUDA command from the “Get Started” selector on pytorch.org. It picks the right CUDA build for you.
  5. Test the GPU from Python: python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))". It should print True and your card name.

Use a separate venv per project. AI projects often pin different torch versions, and mixing them in one environment is the most common source of “it worked yesterday” errors.

Step 5: Run Ollama Inside WSL2

  1. Install: curl -fsSL https://ollama.com/install.sh | sh (the official Linux install command).
  2. Start the server if it is not running already: ollama serve. Leave that terminal open, or let the systemd service run it if your distro has systemd turned on.
  3. Check it: in a second terminal run ollama -v.
  4. Run a model: ollama run llama3.2
  5. Confirm GPU use: run ollama ps while it is loaded. PROCESSOR should say 100% GPU.

New to Ollama? Our complete Ollama guide covers models, commands and the API. If you prefer full control over quantisation and flags, the llama.cpp guide works in WSL2 the same way it does on any Ubuntu machine.

Do not run the Windows and WSL versions of Ollama at the same time. Both try to use port 11434. Pick one, or set OLLAMA_HOST to a different port for the second.

Step 6: Docker With GPU Support

Many AI web UIs ship as Docker images. You have two choices.

Method How Notes
Docker Desktop Install on Windows, turn on the WSL 2 backend and WSL integration for Ubuntu Easiest; GPU works with --gpus all
Docker Engine in Ubuntu curl https://get.docker.com | sh, then sudo service docker start Also install the NVIDIA Container Toolkit
  1. If you use Docker Engine, install the NVIDIA Container Toolkit using the current steps from NVIDIA’s Container Toolkit install guide.
  2. Test GPU access: run a CUDA container with docker run --rm --gpus all plus an NVIDIA CUDA image tag, and call nvidia-smi inside it.
  3. Run your app with --gpus all. NVIDIA notes that under WSL you cannot pick single GPUs by index this way; it is all GPUs or none.

Step 7: Give WSL2 More RAM

By default WSL2 can use 50% of your Windows RAM and all CPU threads, plus a swap file of 25% of RAM. For big models that offload to system memory, raise the limit.

  1. Easiest: open WSL Settings from the Start menu. Microsoft recommends this over editing the file by hand.
  2. Or create the file: in File Explorer go to %UserProfile% and create .wslconfig.
  3. Add these lines (example for a 64 GB PC):
    [wsl2]
    memory=48GB
    processors=16
    swap=16GB
  4. Apply it: run wsl --shutdown in PowerShell, wait about 8 seconds, then open Ubuntu again.
  5. Check: inside Ubuntu run free -h.

Leave at least 8 to 16 GB for Windows itself. Giving WSL all your RAM can make Windows stutter while a model loads.

Where to Store Models and Files

Microsoft’s docs are clear: for the best speed, keep files in the Linux file system when you work from a Linux terminal. Reading a 20 GB model from /mnt/c/... crosses the Windows/Linux boundary and is much slower.

Location Path Speed from Linux
Linux home ~/ (/home/you) Fast
Windows drive /mnt/c/Users/you/... Slow for big files

You can still browse Linux files from Windows: type \\wsl$ in File Explorer, or run explorer.exe . inside Ubuntu.

The Linux disk is a virtual file (ext4.vhdx) that grows as you add models. Delete models you no longer use with ollama rm or by removing the folder.

AMD and Intel GPUs

CUDA in WSL2 is NVIDIA only. For AMD and Intel cards, Microsoft’s WSL GPU guide points to DirectML builds of PyTorch and TensorFlow, which work across AMD, Intel and NVIDIA.

For chatting with local models on an AMD or Intel card, native Windows apps are often the simpler path. Ollama on Windows supports many Radeon cards through ROCm and more GPUs through Vulkan, and LM Studio supports Vulkan too. Our RX 9070 XT for AI guide covers the AMD side in detail.

Troubleshooting

Problem Likely cause Fix
Error 0x80370102 or “virtualization not enabled” VT-x/AMD-V off in BIOS Enable virtualization in BIOS/UEFI, then retry
Install stuck at 0.0% Store download issue wsl --install --web-download -d Ubuntu
nvidia-smi not found PATH or old driver Use /usr/lib/wsl/lib/nvidia-smi; update the Windows driver; wsl --shutdown
GPU vanished after installing CUDA A Linux driver got installed Remove cuda-drivers/nvidia-driver-* packages, reinstall toolkit-only
torch.cuda.is_available() is False CPU-only torch wheel Reinstall using the CUDA command from pytorch.org
Out of memory with big models WSL RAM cap at 50% Raise memory= in .wslconfig
Model loads very slowly Model stored on /mnt/c Move it to your Linux home folder
Port 11434 already in use Windows Ollama also running Quit one, or change OLLAMA_HOST
Some training jobs crash WSL limits on pinned and managed memory Lower batch size, or use native Linux for heavy training
Can’t open web UI at localhost Network mode or app bound to 127.0.0.1 only Bind the app to 0.0.0.0, or try networkingMode=mirrored

My Take

For anyone on Windows with an NVIDIA card who wants to try GitHub AI projects, WSL2 is the best setup. Install it, update the Windows driver, run nvidia-smi, and you are 90% done.

The two rules that save the most pain: never install a Linux NVIDIA driver inside WSL, and keep your models in the Linux home folder. If you mostly chat with models, native Windows apps are still fine, and you can always add WSL2 later.

WSL2 is also a good home for coding agents that expect a Linux shell. See our Claude Code vs Gemini CLI comparison if you want an AI coding assistant running there too.

FAQ

Can WSL2 use my GPU for AI?

Yes. With an NVIDIA card and a current Windows driver, CUDA apps in WSL2 use the GPU. AMD and Intel cards can use DirectML builds of PyTorch and TensorFlow.

Do I need to install NVIDIA drivers in Ubuntu?

No. Install only the Windows driver. Installing a Linux NVIDIA driver inside WSL breaks GPU access.

Is WSL2 slower than native Linux for AI?

Inference speed is usually close. Heavy training can hit WSL limits on pinned and managed memory, and files on /mnt/c load slowly.

How much RAM does WSL2 use?

Up to 50% of your Windows RAM by default. Change it with memory= in .wslconfig or in WSL Settings.

Should I use Ollama on Windows or in WSL2?

For plain chat, the Windows app is simpler. Use WSL2 if your other tools are Linux based, or if you want everything in one Linux environment.

Does WSL2 work on Windows 10?

Yes, on version 2004 (build 19041) or newer. GPU compute needs Windows 10 21H2 or newer, or Windows 11.

How do I uninstall a WSL distro?

Run wsl --unregister Ubuntu in PowerShell. This deletes that distro and all its files, so copy anything you need first.

Sources

Free weekly email

Get new AI guides every week

One short weekly email with our newest local AI, ComfyUI and AI tool guides. No spam, no pop-ups, just the good stuff.

Free. Unsubscribe anytime in one click. We only use your email to send this newsletter. See our privacy policy.

Thrilled woman in a green jacket in a neon-lit tech lab holding a white card reading WSL2, hook text: Linux AI on Windows