AMD GPU for AI in 2026: ROCm on Windows and Linux, Ollama, LM Studio and ComfyUI Setup (RX 9070 XT, 7900 XTX, R9700)

AMD cards give you a lot of VRAM for the money. A Radeon AI PRO R9700 has 32 GB for roughly a quarter of an RTX 5090’s current price, and an RX 9070 XT or used RX 7900 XTX costs far less than NVIDIA’s 16-24 GB cards. The question is always the same: will my AI software actually run on it? In 2026 the answer is mostly yes, much more than two years ago. This guide shows which Radeon GPUs are supported, what works on Windows vs Linux, step-by-step setup for Ollama, LM Studio, llama.cpp and ComfyUI, the real speed gap to NVIDIA, and fixes for the usual errors.

AMD for AI in 2026: The Short Version

NVIDIA’s CUDA is still the default target for AI developers, so new tools land there first. AMD’s answer is ROCm, its open compute stack, plus the HIP layer that lets CUDA-style code run on Radeon. Three things changed in 2026:

• ComfyUI Desktop got official ROCm support on Windows in January 2026 (v0.7.0).
• AMD Software: PyTorch on Windows (ROCm 7.2.1, March 2026) gave Windows users a supported PyTorch build for RDNA 3/4 cards and Ryzen AI APUs.
• Vulkan backends in llama.cpp, LM Studio and Ollama matured, so GGUF LLMs run well on almost any modern Radeon without ROCm at all.

Supported Radeon GPUs and Ryzen APUs

“Not listed” doesn’t always mean “won’t work”, but you’ll be on your own with workarounds. If you’re buying, stick to a listed card.

What Works and What Doesn’t

Setup: LM Studio (Easiest)

1. Update to the latest AMD Software: Adrenalin Edition driver.
2. Install LM Studio from lmstudio.ai.
3. Open Settings > Runtime and choose the Vulkan llama.cpp runtime (or the ROCm runtime if your card is listed).
4. Download a GGUF model sized for your VRAM (for a 16 GB card, a 14B Q4_K_M or a 20B-24B Q3/Q4).
5. When loading, set GPU offload to the maximum layers that fit.

If you’re new to this, my Ollama vs LM Studio vs llama.cpp comparison explains which front end suits you.

Setup: Ollama

1. Install the latest Adrenalin driver (Windows) or ROCm packages (Linux).
2. Install Ollama from ollama.com (Windows installer, or curl -fsSL https://ollama.com/install.sh | sh on Linux, which pulls in the ROCm build when it detects a Radeon).
3. Run a model: ollama run qwen3:14b.
4. Check it’s on the GPU: ollama ps should show 100% GPU.
5. If your card isn’t detected by ROCm, enable the Vulkan backend: set the environment variable OLLAMA_VULKAN=1 and restart Ollama.

On Linux, some older RDNA 3 cards that aren’t on the official list work with HSA_OVERRIDE_GFX_VERSION=11.0.0. Use it only if you know your card’s gfx family. More detail in how to install Ollama and fix it not using the GPU.

Setup: llama.cpp

Prebuilt Windows and Linux releases include Vulkan builds. To build for ROCm/HIP on Linux:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_HIP=ON
cmake --build build --config Release -j

Or for Vulkan: cmake -B build -DGGML_VULKAN=ON. Then serve a model with llama serve -m model.gguf -ngl 99 (or the older llama-server binary on older builds). Try both backends: on some cards Vulkan is faster for token generation while HIP is faster for prompt processing.

Setup: ComfyUI on Windows and Linux

Windows: ComfyUI Desktop (recommended)

1. Install Adrenalin 26.2.2 or newer, then the PyTorch on Windows component AMD links from the release notes.
2. Download ComfyUI Desktop from comfy.org/download.
3. During setup, AMD ROCm is selected automatically on supported GPUs. Finish the install.
4. Load a starter workflow (SDXL or a FLUX-class model) and run it.
5. On 8-16 GB cards, if you run out of memory, add --lowvram --disable-pinned-memory to the launch arguments, as AMD suggests. My ComfyUI command line arguments guide explains where to put them.

Linux: ComfyUI from Git

1. Install ROCm for your distro from AMD’s docs (Ubuntu 24.04 or 22.04 is the smoothest).
2. Add your user to the GPU groups: sudo usermod -aG render,video $USER, then log out and in.
3. Clone ComfyUI and create a virtual environment.
4. Install the ROCm build of PyTorch using the exact command from the pytorch.org Get Started selector (choose Linux, Pip, ROCm). It uses an index URL of the form https://download.pytorch.org/whl/rocmX.Y.
5. pip install -r requirements.txt, then python main.py.
6. Check the console: it should list your Radeon as the device.

Tip: GGUF models through ComfyUI-GGUF have been reported to run slowly on some Radeon cards; if that happens, try FP8 or safetensors versions instead.

Performance vs NVIDIA

Independent testing published in April 2026 found a consistent 30-40% gap between an RX 7900 XTX and an RTX 4090 for LLM inference, largely because RDNA 3 lacks NVIDIA-style tensor cores. On Llama 3.1 8B Q4_K_M, the 4090 managed about 158 tokens/s vs 119 tokens/s on the 7900 XTX, and more than twice the prompt processing speed. ComfyUI image generation was 25-40% slower on AMD.

RDNA 4 (RX 9070 XT, R9700) has much stronger AI hardware than RDNA 3, which narrows the gap for FP8 and INT8 work, but software support for those paths is still catching up. The fair way to think about it: AMD sells you more memory per dollar, NVIDIA sells you more speed and fewer headaches.

Which AMD Card to Buy for AI

• Radeon AI PRO R9700 (32 GB): the cheapest new 32 GB card, about $1,700-$1,850. Ideal for 27B-32B LLMs and bigger image models. Blower cooler, so it’s loud but stacks well.
• RX 9070 XT (16 GB): good all-rounder for gaming plus 8B-20B LLMs and image generation.
• Used RX 7900 XTX (24 GB): excellent memory per dollar, fully supported on Windows and Linux.
• Ryzen AI Max+ 395 mini PCs and laptops: up to 128 GB of unified memory for large MoE models, slower but huge capacity. See Mac vs PC for local AI.

For the full NVIDIA vs AMD buying picture, see the best GPU for local LLMs and the best workstation GPU for AI.

Troubleshooting

Ollama runs on the CPU

Run ollama ps. If it shows CPU, update the driver, restart Ollama, then try OLLAMA_VULKAN=1. On Linux check rocminfo lists your GPU.

“HIP error: invalid device function”

Your PyTorch or llama.cpp build wasn’t compiled for your card’s gfx target. Install the build for your ROCm version, or on Linux set HSA_OVERRIDE_GFX_VERSION to the nearest supported family.

ComfyUI says “Torch not compiled with CUDA enabled”

A custom node is calling CUDA directly. Update the node, look for an AMD-compatible fork, or replace it. Core ComfyUI nodes don’t need CUDA.

Black screen or driver timeout during long generations

Update Adrenalin, disable overclocks and undervolts while testing, and on Windows make sure you installed the PyTorch on Windows package that matches your driver version.

rocminfo shows nothing on Linux

Add your user to the render and video groups, reboot, and confirm the amdgpu kernel module is loaded with lsmod | grep amdgpu.

Slow speeds on a Ryzen AI Max laptop

Raise the GPU memory allocation (often called UMA frame buffer or VRAM size) in the BIOS or AMD Software, plug into power, and pick the performance power mode.

Frequently Asked Questions

Can I use an AMD GPU for AI in 2026?

Yes. LM Studio, Ollama, llama.cpp and ComfyUI all run on supported Radeon cards, on Windows 11 and Linux. A few CUDA-only tools and custom nodes still don’t.

Does ROCm work on Windows now?

Yes. AMD’s PyTorch on Windows package based on ROCm 7.2.1 supports RX 9070, 9070 XT, 9060 XT, R9700, RX 7900 XTX, W7900, RX 7700 and several Ryzen AI APUs on Windows 11.

Is the RX 9070 XT good for AI?

It’s a solid 16 GB card for 8B-20B LLMs and image generation, with official ROCm support on Windows and Linux. It’s slower than an RTX 5070 Ti or 5080 in many AI tasks but usually cheaper.

Does Ollama support AMD GPUs?

Yes. Ollama uses ROCm on supported Radeon cards and can use Vulkan on others by setting OLLAMA_VULKAN=1.

How much slower is AMD than NVIDIA for AI?

In 2026 tests an RX 7900 XTX was roughly 25-40% slower than an RTX 4090 for LLM inference and ComfyUI. RDNA 4 cards close some of that gap.

Does ComfyUI work on AMD GPUs?

Yes. ComfyUI Desktop has official ROCm support on Windows since v0.7.0, and ComfyUI runs on Linux with the ROCm build of PyTorch. Some CUDA-only custom nodes may not work.

Is the Radeon AI PRO R9700 worth it for local LLMs?

For memory per dollar, yes. It’s the cheapest new 32 GB card, which fits 27B-32B models fully on the GPU.

Final Thoughts

If you mainly run LLMs, an AMD card is an easy recommendation in 2026: LM Studio or Ollama with Vulkan or ROCm just works, and you get more VRAM per dollar. For ComfyUI it now works out of the box on Windows, but expect slower generations and the occasional custom node that refuses to cooperate. If your workflow depends on niche CUDA tools, NVIDIA is still the safer buy.

Before buying, check how much VRAM you need, and once set up, use these commands to check VRAM usage (amd-smi and rocm-smi included).

Sources

• AMD ROCm docs: Windows support matrix (ROCm 7.2.1)
• AMD: PyTorch on Windows Edition 7.2.1 release notes (26 Mar 2026)
• Comfy blog: Official AMD ROCm support arrives on Windows for ComfyUI Desktop (6 Jan 2026)
• IDFS AI: AMD GPUs for AI inference in 2026 (benchmarks, 14 Apr 2026)
• ROCm Blogs: ComfyUI on Radeon RX 9000 series