Skip to content
,

How to Run gpt-oss Locally: OpenAI’s Open Model on Ollama, LM Studio and llama.cpp (2026)

7–10 minutes
Smiling woman holding an OpenAI logo card with the text OpenAI model on your PC, for a guide to running gpt-oss locally

gpt-oss is OpenAI’s open-weight model family. You can download it for free, run it on your own PC with no internet, and use it commercially under the Apache 2.0 licence.

The smaller gpt-oss-20b runs on a single 16 GB GPU or a 16 GB+ Mac, and gives you a ChatGPT-style reasoning model at home. The bigger gpt-oss-120b needs about 80 GB of memory.

This guide shows how to run both with Ollama, LM Studio and llama.cpp, how much VRAM you need, how to set reasoning effort, how to use tools and the API, and how to fix the usual problems.

Facts come from OpenAI’s Hugging Face model cards, the Ollama library page and the ggml-org GGUF card, checked in November 2026.

TL;DR

  • Models: gpt-oss-20b (21B total, 3.6B active) and gpt-oss-120b (117B total, 5.1B active). Both are mixture-of-experts, Apache 2.0, 128K context.
  • Memory: 20b fits in about 16 GB (Ollama download 14 GB). 120b fits on one 80 GB GPU (download 65 GB).
  • Easiest: ollama run gpt-oss:20b
  • LM Studio: lms get openai/gpt-oss-20b or search “gpt-oss” in the app.
  • llama.cpp: llama serve -hf ggml-org/gpt-oss-20b-GGUF (older builds: llama-server).
  • Reasoning: set low, medium or high, for example Reasoning: high in the system prompt.
  • Newest versions: no gpt-oss-2 yet. The only newer releases are the gpt-oss-safeguard safety models.

What Is gpt-oss?

OpenAI released gpt-oss in August 2025 as its first open-weight language models since GPT-2. They are reasoning models built for agent tasks, tool use and coding.

Model Total params Active params Context Memory needed
gpt-oss-20b 21B 3.6B 128K About 16 GB
gpt-oss-120b 117B 5.1B 128K One 80 GB GPU (H100, MI300X) or equivalent
  • Mixture-of-experts: only a few billion parameters work on each token, so it runs fast for its size.
  • MXFP4 weights: the expert weights (over 90% of parameters) ship at about 4.25 bits each. That is why the 20b fits in 16 GB without extra quantising.
  • Apache 2.0: free for personal and commercial use.
  • Harmony format: the models need OpenAI’s “harmony” chat format. Ollama, LM Studio, llama.cpp and the Transformers chat template handle it for you.
  • Tools: function calling, web browsing, Python code execution and structured outputs.

Is there a newer version? As of November 2026, OpenAI’s Hugging Face page shows no gpt-oss-2. The newer uploads are gpt-oss-safeguard-20b and gpt-oss-safeguard-120b, which are safety classifier models for checking content against your policies, not general chat upgrades.

Hardware Requirements

Setup gpt-oss-20b gpt-oss-120b
NVIDIA GPU 16 GB VRAM (RTX 4060 Ti 16GB, 4080, 5080, 5070 Ti); 12 GB works with some CPU offload 80 GB VRAM, or 2+ big GPUs, or GPU plus lots of RAM (slow)
Mac 16 GB unified memory minimum; 24 GB+ is comfortable 96 GB to 128 GB unified memory
AMD Strix Halo / 128 GB systems Easy Fits in unified memory
CPU only Works with 32 GB RAM, slowly Not practical
Disk About 14 GB About 65 GB

Not sure what your card can handle? See our VRAM guide for AI. For a 128 GB unified-memory option, our Ryzen AI Max+ 395 (Strix Halo) guide covers running big MoE models like the 120b.

Method 1: Ollama

  1. Install Ollama from ollama.com (Windows, Mac or Linux).
  2. Download and run the 20b: ollama run gpt-oss:20b. The first run downloads about 14 GB.
  3. Or the 120b on big hardware: ollama run gpt-oss:120b (about 65 GB).
  4. Check GPU use: run ollama ps. You want 100% GPU. A CPU/GPU split means it did not fully fit.
  5. Use the app or API: the Ollama desktop app has a chat window, and the API is at http://localhost:11434.

Ollama runs MXFP4 natively and offers optional built-in web search, which is off by default. If generation is slow, our Ollama not using GPU fix walks through every cause. New to Ollama? Start with our complete Ollama guide.

Method 2: LM Studio

  1. Install LM Studio from lmstudio.ai.
  2. Search “gpt-oss” in the model search and download openai/gpt-oss-20b. Or in a terminal: lms get openai/gpt-oss-20b.
  3. Load the model and set GPU offload to max if it fits in VRAM.
  4. Pick reasoning effort in the chat settings where LM Studio shows it for gpt-oss, or add Reasoning: high to the system prompt.
  5. Optional: start the local server (port 1234) to use it from other apps.

Our LM Studio beginner guide explains GPU offload, context length and the local server in detail.

Method 3: llama.cpp

  1. Get llama.cpp: download a release build for your GPU, or build it from source.
  2. Start a server with web UI: llama serve -hf ggml-org/gpt-oss-20b-GGUF. It downloads the model from Hugging Face automatically.
  3. Or chat in the terminal: llama cli -hf ggml-org/gpt-oss-20b-GGUF
  4. Older builds use llama-server and llama-cli with the same -hf argument.
  5. Open the web UI at the address it prints (usually http://127.0.0.1:8080), which also serves an OpenAI-compatible API.

The ggml-org GGUF keeps the native MXFP4 weights (about 12 GB for the 20b). Our llama.cpp guide covers the new command names, GPU layers and server flags.

Method 4: vLLM and Transformers (Advanced)

  • vLLM (Linux, NVIDIA/AMD data-centre or high-end GPUs): vllm serve openai/gpt-oss-20b. Gives an OpenAI-compatible server with high throughput for many users.
  • Transformers (Python): pip install -U transformers kernels torch, then load openai/gpt-oss-20b. The chat template applies the harmony format. If you call model.generate yourself, apply harmony with the openai-harmony package.
  • Docker Model Runner: docker model run hf.co/openai/gpt-oss-20b

Windows users who want vLLM can run it inside WSL2. Our WSL2 local AI setup guide shows the GPU setup.

Set Reasoning Effort

gpt-oss thinks before it answers. You choose how hard it thinks, trading speed for quality.

Level How to set Use for
Low Reasoning: low in the system prompt Chat, quick facts, simple rewrites
Medium (default) Reasoning: medium Everyday tasks, balanced speed
High Reasoning: high Maths, coding, multi-step logic

The full chain of thought is visible, which helps with debugging. OpenAI says it is not meant to be shown to end users, so hide it if you build an app for others.

Tools, Web Search and the API

  • Function calling: define tools with JSON schemas and the model calls them, just like OpenAI’s API.
  • Structured outputs: ask for JSON that matches a schema.
  • Web search: Ollama has an optional built-in web search for gpt-oss; other apps can connect search through tools or MCP.
  • Code execution: the model was trained to use a Python tool, which agent frameworks can provide.
  • OpenAI-compatible API: point any OpenAI SDK at your local server (Ollama http://localhost:11434/v1, LM Studio http://localhost:1234/v1, llama.cpp port 8080) and use the model name.

Because the API shape matches OpenAI’s, many apps built for GPT work with gpt-oss after you change the base URL. If you also want cloud GPT models, see how to get an OpenAI API key.

gpt-oss vs Other Local Models

gpt-oss-20b Typical dense 14B Typical 30B MoE
Memory (4-bit) About 14 GB About 9 GB About 18 GB
Speed Fast (3.6B active) Medium Fast
Strengths Reasoning, tool use, OpenAI-style behaviour General chat Broad knowledge
Licence Apache 2.0 Varies Varies
Image input No (text only) Some models Some models

gpt-oss is text only. For image understanding, pick a vision model instead. Newer open models from other labs may beat it on some benchmarks, so test it on your own tasks.

Troubleshooting

Problem Likely cause Fix
Very slow replies Model partly on CPU Check ollama ps; close other GPU apps; lower context
Out of memory loading 20b Under 16 GB free VRAM/RAM Allow CPU offload, reduce context, or close apps
Strange tags or broken output Harmony format not applied Use an up-to-date Ollama, LM Studio or llama.cpp; use the chat template in Transformers
It “thinks” for too long High reasoning effort Set Reasoning: low or medium
Tool calls fail in my app App not passing tool schema correctly Test with the server’s OpenAI-compatible endpoint and a simple tool first
Download stalls Network or disk space Free 15 GB+ (20b) or 70 GB+ (120b) and retry
120b crawls on a 24 GB GPU Most of it runs from system RAM Use the 20b, or hardware with 80 GB+ fast memory
Command llama serve not found Older llama.cpp build Use llama-server, or update llama.cpp

My Take

For anyone with a 16 GB GPU or a 16 GB+ Mac, gpt-oss-20b is still one of the best “first serious local model” choices.

It is fast thanks to MoE, it reasons well on high effort, tool calling works out of the box, and the Apache 2.0 licence removes any worry about using it in projects.

Start with ollama run gpt-oss:20b, try medium and high reasoning, and only chase the 120b if you have 80 GB+ of fast memory.

FAQ

Is gpt-oss free?

Yes. Both models are free to download and use, including commercially, under the Apache 2.0 licence.

Can I run gpt-oss on a 16 GB GPU?

Yes, the 20b is designed to run in about 16 GB of memory thanks to its MXFP4 weights.

Can I run gpt-oss-120b at home?

Only with about 80 GB of fast memory: a data-centre GPU, several large GPUs, or a 96 GB to 128 GB unified-memory machine.

Is gpt-oss the same as ChatGPT?

No. It is a separate open-weight model from OpenAI. It behaves in a similar style, but it is not the model behind ChatGPT.

Does gpt-oss work offline?

Yes. After download it runs fully offline. Web search only works if you turn on a search tool.

What context length does gpt-oss support?

Up to 128K tokens. Longer context uses more memory, so many apps default lower.

Can gpt-oss see images?

No. Both models take text input only.

Is there a gpt-oss 2?

Not as of November 2026. OpenAI’s newer open releases are the gpt-oss-safeguard safety models.

Sources

Free weekly email

Get new AI guides every week

One short weekly email with our newest local AI, ComfyUI and AI tool guides. No spam, no pop-ups, just the good stuff.

Free. Unsubscribe anytime in one click. We only use your email to send this newsletter. See our privacy policy.

Smiling woman holding an OpenAI logo card with the text OpenAI model on your PC, for a guide to running gpt-oss locally