Lots of local AI tools are built for Linux first. Install scripts, CUDA builds, Python packages and GitHub projects often assume Ubuntu, and Windows users end up fighting missing wheels and path errors.
WSL2 (Windows Subsystem for Linux) fixes that. It runs a real Ubuntu system inside Windows, and with an NVIDIA card it can use your GPU through CUDA at near-native speed.
This guide takes you from a clean Windows PC to a working local AI setup in WSL2: install, GPU check, CUDA, Python, Ollama, Docker, memory limits and fixes for the common errors. Every command comes from Microsoft’s WSL docs, NVIDIA’s CUDA on WSL guide and Ollama’s Linux docs, checked in November 2026.
- Install: open PowerShell as admin, run
wsl --install, restart, then create a Linux user. - GPU: install only the normal NVIDIA Windows driver. Never install a Linux NVIDIA driver inside WSL.
- Check: run
nvidia-smiinside Ubuntu. If it lists your card, the GPU is ready. - CUDA toolkit: only needed to compile code. Use NVIDIA’s WSL-Ubuntu installer or the
cuda-toolkitpackage, nevercudaorcuda-drivers. - Ollama:
curl -fsSL https://ollama.com/install.sh | shinside Ubuntu. - Speed: keep models and projects in your Linux home folder, not on
/mnt/c. - RAM: WSL2 gets 50% of your RAM by default. Raise it in
%UserProfile%\.wslconfigor WSL Settings.
Why Use WSL2 for Local AI?
Native Windows apps like LM Studio and the Ollama Windows app are fine for simple chat. WSL2 starts to matter when you go beyond that.
- Linux-first projects: many GitHub AI repos only document Linux setup, and some Python packages ship Linux CUDA wheels first.
- Same commands as tutorials: most guides use bash, apt and Linux paths, so you can copy them as written.
- GPU access: WSL2 passes your NVIDIA GPU through to Linux, so CUDA apps run on the card.
- No dual boot: you keep Windows for games and apps and open Ubuntu in a terminal tab.
- Clean sandbox: you can break and reinstall a Linux distro without touching Windows.
| Option | Best for | Trade-off |
|---|---|---|
| Native Windows apps | Simple chat, one-click tools | Some projects lack Windows support |
| WSL2 (Ubuntu) | Python projects, servers, Linux-only tools | Some limits on pinned and managed GPU memory |
| Dual boot Linux | Maximum performance, heavy training | Restart to switch, more setup |
Requirements
| Item | Requirement |
|---|---|
| Windows | Windows 11, or Windows 10 version 2004+ (build 19041+) for wsl --install. GPU compute needs Windows 11 or Windows 10 21H2+. |
| Virtualization | Enabled in BIOS/UEFI (Intel VT-x or AMD-V/SVM) |
| NVIDIA GPU | Pascal (GTX 10-series) or newer recommended; Maxwell is not officially supported |
| Driver | Latest NVIDIA Windows display driver (GeForce or RTX/Quadro) |
| Disk | At least 50 GB free; models add 4 to 50+ GB each |
| RAM | 16 GB minimum, 32 GB+ recommended for bigger models |
Step 1: Install WSL2 and Ubuntu
- Open PowerShell as administrator: right-click Start, choose Terminal (Admin) or Windows PowerShell (Admin).
- Run the installer:
wsl --install. This turns on the features WSL needs and installs Ubuntu. - Restart your PC when it asks.
- Open Ubuntu from the Start menu. The first launch unpacks files, then asks you to create a Linux username and password. This password is for
sudoinside Linux only. - Update WSL itself: in PowerShell run
wsl --updateto get the latest kernel. - Confirm the version: run
wsl -l -v. The VERSION column must say 2. If it says 1, runwsl --set-version Ubuntu 2. - Update Ubuntu packages: inside Ubuntu run
sudo apt update && sudo apt upgrade -y.
If wsl --install only prints help text, WSL is already present. Run wsl --list --online to see distros, then wsl --install -d Ubuntu. If the install hangs at 0.0%, Microsoft’s fix is wsl --install --web-download -d Ubuntu.
Step 2: Give WSL2 Your NVIDIA GPU
This is the step most people get wrong. NVIDIA is clear that the Windows driver is the only driver you need. It is exposed inside WSL2 as a stub library, libcuda.so, so Linux apps can talk to the card.
- Install or update the NVIDIA driver on Windows from nvidia.com or the NVIDIA app. A normal Game Ready or Studio driver works.
- Do not install any Linux NVIDIA driver inside Ubuntu. Skip
apt install nvidia-driver-*and the.rundriver installers. - Restart WSL after a driver update: in PowerShell run
wsl --shutdown, then open Ubuntu again. - Test it: inside Ubuntu run
nvidia-smi. You should see your GPU name, driver version and VRAM.
If nvidia-smi is “not found”, try /usr/lib/wsl/lib/nvidia-smi. NVIDIA notes it lives there in WSL, and you can add /usr/lib/wsl/lib to your PATH. Note that some GPU stats, like utilisation and the process list, are not supported inside WSL, so blank fields there are normal.
Step 3: Install the CUDA Toolkit (Optional)
You do not need the full CUDA toolkit to run Ollama or to use PyTorch from pip, because those ship their own CUDA runtime. You need it only if you compile CUDA code yourself, for example building llama.cpp from source or some custom kernels.
- Remove NVIDIA’s old signing key:
sudo apt-key del 7fa2af80 - Open the CUDA Toolkit download page on developer.nvidia.com and choose Linux, x86_64, WSL-Ubuntu, then a deb installer.
- Run the commands it shows. The WSL-Ubuntu package leaves out the Linux GPU driver, so it cannot overwrite the WSL driver.
- If you use apt meta-packages, install only
cuda-toolkit-12-x(or the matching toolkit-only package for your version). - Check it:
nvcc --version. If it is not found, add/usr/local/cuda/binto your PATH in~/.bashrc.
Warning: do not install the cuda, cuda-12-x or cuda-drivers meta-packages. NVIDIA says these try to install a Linux driver, which breaks GPU access in WSL.
Step 4: Set Up Python for AI
- Install basics:
sudo apt install -y python3-venv python3-pip git build-essential - Make a project folder in Linux:
mkdir -p ~/ai && cd ~/ai - Create a virtual environment:
python3 -m venv .venv && source .venv/bin/activate - Install PyTorch with CUDA: copy the Linux + pip + CUDA command from the “Get Started” selector on pytorch.org. It picks the right CUDA build for you.
- Test the GPU from Python:
python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))". It should printTrueand your card name.
Use a separate venv per project. AI projects often pin different torch versions, and mixing them in one environment is the most common source of “it worked yesterday” errors.
Step 5: Run Ollama Inside WSL2
- Install:
curl -fsSL https://ollama.com/install.sh | sh(the official Linux install command). - Start the server if it is not running already:
ollama serve. Leave that terminal open, or let the systemd service run it if your distro has systemd turned on. - Check it: in a second terminal run
ollama -v. - Run a model:
ollama run llama3.2 - Confirm GPU use: run
ollama pswhile it is loaded. PROCESSOR should say 100% GPU.
New to Ollama? Our complete Ollama guide covers models, commands and the API. If you prefer full control over quantisation and flags, the llama.cpp guide works in WSL2 the same way it does on any Ubuntu machine.
Do not run the Windows and WSL versions of Ollama at the same time. Both try to use port 11434. Pick one, or set OLLAMA_HOST to a different port for the second.
Step 6: Docker With GPU Support
Many AI web UIs ship as Docker images. You have two choices.
| Method | How | Notes |
|---|---|---|
| Docker Desktop | Install on Windows, turn on the WSL 2 backend and WSL integration for Ubuntu | Easiest; GPU works with --gpus all |
| Docker Engine in Ubuntu | curl https://get.docker.com | sh, then sudo service docker start |
Also install the NVIDIA Container Toolkit |
- If you use Docker Engine, install the NVIDIA Container Toolkit using the current steps from NVIDIA’s Container Toolkit install guide.
- Test GPU access: run a CUDA container with
docker run --rm --gpus allplus an NVIDIA CUDA image tag, and callnvidia-smiinside it. - Run your app with
--gpus all. NVIDIA notes that under WSL you cannot pick single GPUs by index this way; it is all GPUs or none.
Step 7: Give WSL2 More RAM
By default WSL2 can use 50% of your Windows RAM and all CPU threads, plus a swap file of 25% of RAM. For big models that offload to system memory, raise the limit.
- Easiest: open WSL Settings from the Start menu. Microsoft recommends this over editing the file by hand.
- Or create the file: in File Explorer go to
%UserProfile%and create.wslconfig. - Add these lines (example for a 64 GB PC):
[wsl2] memory=48GB processors=16 swap=16GB - Apply it: run
wsl --shutdownin PowerShell, wait about 8 seconds, then open Ubuntu again. - Check: inside Ubuntu run
free -h.
Leave at least 8 to 16 GB for Windows itself. Giving WSL all your RAM can make Windows stutter while a model loads.
Where to Store Models and Files
Microsoft’s docs are clear: for the best speed, keep files in the Linux file system when you work from a Linux terminal. Reading a 20 GB model from /mnt/c/... crosses the Windows/Linux boundary and is much slower.
| Location | Path | Speed from Linux |
|---|---|---|
| Linux home | ~/ (/home/you) |
Fast |
| Windows drive | /mnt/c/Users/you/... |
Slow for big files |
You can still browse Linux files from Windows: type \\wsl$ in File Explorer, or run explorer.exe . inside Ubuntu.
The Linux disk is a virtual file (ext4.vhdx) that grows as you add models. Delete models you no longer use with ollama rm or by removing the folder.
AMD and Intel GPUs
CUDA in WSL2 is NVIDIA only. For AMD and Intel cards, Microsoft’s WSL GPU guide points to DirectML builds of PyTorch and TensorFlow, which work across AMD, Intel and NVIDIA.
For chatting with local models on an AMD or Intel card, native Windows apps are often the simpler path. Ollama on Windows supports many Radeon cards through ROCm and more GPUs through Vulkan, and LM Studio supports Vulkan too. Our RX 9070 XT for AI guide covers the AMD side in detail.
Troubleshooting
| Problem | Likely cause | Fix |
|---|---|---|
| Error 0x80370102 or “virtualization not enabled” | VT-x/AMD-V off in BIOS | Enable virtualization in BIOS/UEFI, then retry |
| Install stuck at 0.0% | Store download issue | wsl --install --web-download -d Ubuntu |
nvidia-smi not found |
PATH or old driver | Use /usr/lib/wsl/lib/nvidia-smi; update the Windows driver; wsl --shutdown |
| GPU vanished after installing CUDA | A Linux driver got installed | Remove cuda-drivers/nvidia-driver-* packages, reinstall toolkit-only |
torch.cuda.is_available() is False |
CPU-only torch wheel | Reinstall using the CUDA command from pytorch.org |
| Out of memory with big models | WSL RAM cap at 50% | Raise memory= in .wslconfig |
| Model loads very slowly | Model stored on /mnt/c |
Move it to your Linux home folder |
| Port 11434 already in use | Windows Ollama also running | Quit one, or change OLLAMA_HOST |
| Some training jobs crash | WSL limits on pinned and managed memory | Lower batch size, or use native Linux for heavy training |
| Can’t open web UI at localhost | Network mode or app bound to 127.0.0.1 only | Bind the app to 0.0.0.0, or try networkingMode=mirrored |
My Take
For anyone on Windows with an NVIDIA card who wants to try GitHub AI projects, WSL2 is the best setup. Install it, update the Windows driver, run nvidia-smi, and you are 90% done.
The two rules that save the most pain: never install a Linux NVIDIA driver inside WSL, and keep your models in the Linux home folder. If you mostly chat with models, native Windows apps are still fine, and you can always add WSL2 later.
WSL2 is also a good home for coding agents that expect a Linux shell. See our Claude Code vs Gemini CLI comparison if you want an AI coding assistant running there too.
FAQ
Can WSL2 use my GPU for AI?
Yes. With an NVIDIA card and a current Windows driver, CUDA apps in WSL2 use the GPU. AMD and Intel cards can use DirectML builds of PyTorch and TensorFlow.
Do I need to install NVIDIA drivers in Ubuntu?
No. Install only the Windows driver. Installing a Linux NVIDIA driver inside WSL breaks GPU access.
Is WSL2 slower than native Linux for AI?
Inference speed is usually close. Heavy training can hit WSL limits on pinned and managed memory, and files on /mnt/c load slowly.
How much RAM does WSL2 use?
Up to 50% of your Windows RAM by default. Change it with memory= in .wslconfig or in WSL Settings.
Should I use Ollama on Windows or in WSL2?
For plain chat, the Windows app is simpler. Use WSL2 if your other tools are Linux based, or if you want everything in one Linux environment.
Does WSL2 work on Windows 10?
Yes, on version 2004 (build 19041) or newer. GPU compute needs Windows 10 21H2 or newer, or Windows 11.
How do I uninstall a WSL distro?
Run wsl --unregister Ubuntu in PowerShell. This deletes that distro and all its files, so copy anything you need first.




