Skip to content
,

Ollama Context Length: How to Check and Increase num_ctx (2026)

8–12 minutes
Woman in a cosy home office holding a tablet showing the Ollama logo, hook 4K is too small, fix Ollama

Your local model forgets the start of a long chat, ignores half of the PDF you pasted, or a coding agent loses track of files after a few steps. With Ollama, the cause is very often the context length.

Ollama picks a default based on your VRAM, and on most home GPUs that default is only 4,096 tokens, far less than the model can handle.

This guide shows how to check your current context length, every way to change it (app, environment variable, CLI, Modelfile, API), how much extra memory each size costs, and how to fit a bigger context without spilling onto the CPU.

TL;DR

  • Default: 4k tokens under 24 GB VRAM, 32k from 24-48 GB, 256k at 48 GB or more.
  • Check it: run ollama ps and read the CONTEXT column.
  • Change it for everything: set OLLAMA_CONTEXT_LENGTH=32768 and restart Ollama, or use the slider in the Ollama app settings.
  • Change it for one model: PARAMETER num_ctx 32768 in a Modelfile, or "options": {"num_ctx": 32768} in the API.
  • Agents and coding tools: Ollama recommends at least 64,000 tokens. Use OLLAMA_KV_CACHE_TYPE=q8_0 to roughly halve the memory cost.

New to Ollama? My Ollama guide covers install, models, commands and fixes in one place.

What Context Length Means

The context is everything the model can “see” at once: the system prompt, the chat history, any files or search results you paste in, and the answer it is writing. It is measured in tokens; one token is roughly three quarters of an English word.

When a conversation grows past the context length, Ollama has to drop the oldest part to make room. The model then simply does not know what was said earlier.

That is why a small context feels like the model has a short memory, and why it breaks tools that send long instructions, such as coding agents.

Two different numbers matter:

  • The model’s maximum context: what it was trained for, for example 128k or 256k tokens. Run ollama show <model> and look at the “context length” line.
  • The context Ollama actually allocates: what you set (or the default). This is what decides memory use and what the model can see.

Ollama’s Default Context Length

According to Ollama’s current documentation, the default depends on how much VRAM you have:

Your VRAM Default context Typical GPUs
Under 24 GiB 4k (4,096 tokens) RTX 3060, 4060, 4070, 5070, 5080, most laptops
24-48 GiB 32k RTX 3090, 4090, 5090, two 16 GB cards
48 GiB or more 256k Workstation cards, large unified-memory Macs

Cloud models on Ollama run at their maximum context by default. Older guides say the default is 2,048 tokens; that was true for early Ollama versions and is still listed in some parts of the docs, but the VRAM-based defaults above are what current versions use.

For most people on a 12-16 GB card, that means a 4k context out of the box. A 4k window is fine for short chats, but too small for pasting documents, RAG, or agents.

How to Check Your Current Context

Load a model (send it any message), then run:

ollama ps

You will see something like:

NAME           ID              SIZE      PROCESSOR    CONTEXT    UNTIL
qwen3:14b      a1b2c3d4e5f6    13 GB     100% GPU     32768      4 minutes from now
  • CONTEXT is the context length Ollama allocated for this model.
  • PROCESSOR shows where the model runs. 100% GPU is what you want. A split such as 25%/75% CPU/GPU means part of it spilled into system RAM, which is much slower. A bigger context is the most common reason for that split.

Every Way to Change It

Method Applies to Lasts Best for
Ollama app slider All models Until changed Desktop app users
OLLAMA_CONTEXT_LENGTH All models (default) Until changed Servers, Docker, apps that cannot set options
/set parameter num_ctx Current ollama run session Until you exit Quick tests
Modelfile PARAMETER num_ctx One custom model Permanent A model you always want with a big context
API options.num_ctx One request That request Your own scripts and apps

When more than one is set, the most specific wins: a request’s num_ctx beats the model’s Modelfile value, which beats the server default.

Method 1: OLLAMA_CONTEXT_LENGTH (All Models)

This changes the default for every model. It is the most reliable fix for tools that talk to Ollama but cannot send their own settings.

Ollama app: open Settings and move the context length slider. That is the same setting.

Windows:

  1. Quit Ollama from the system tray (right-click the icon, Quit).
  2. Open Start, search “environment variables”, and choose “Edit environment variables for your account”.
  3. Click New. Name: OLLAMA_CONTEXT_LENGTH. Value: 32768 (or your choice).
  4. Click OK, then start Ollama again from the Start menu.

macOS (app):

launchctl setenv OLLAMA_CONTEXT_LENGTH 32768

Then quit and reopen the Ollama app.

Linux (systemd service):

sudo systemctl edit ollama.service

Add these lines, save, then reload and restart:

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=32768"
sudo systemctl daemon-reload
sudo systemctl restart ollama

One-off from a terminal:

OLLAMA_CONTEXT_LENGTH=32768 ollama serve

Docker: add -e OLLAMA_CONTEXT_LENGTH=32768 to your docker run command, or an environment: entry in Compose. See my Ollama Docker guide.

The same method sets every other Ollama variable. My Ollama commands cheat sheet lists them all.

Method 2: Inside ollama run

For a quick test, start a chat and type:

ollama run qwen3:14b
>>> /set parameter num_ctx 32768

The model reloads with the new context. It only lasts for this session. To keep it, save the session as a new model:

>>> /save qwen3-14b-32k

Next time, run ollama run qwen3-14b-32k.

Method 3: A Modelfile (One Model, Permanent)

This is the cleanest way to have one model always load with a large context, while other models keep the default. Create a text file named Modelfile:

FROM qwen3:14b
PARAMETER num_ctx 32768

Then build the new model:

ollama create qwen3-14b-32k -f Modelfile

This does not copy the weights; it reuses the existing model and only stores the new settings, so it takes seconds and almost no disk space. Use the new name in any app. My Ollama Modelfile guide covers every other parameter you can set the same way.

Method 4: The API and Apps

With Ollama’s native API, send num_ctx in options:

curl http://localhost:11434/api/chat -d '{
  "model": "qwen3:14b",
  "messages": [{"role": "user", "content": "Summarise this report..."}],
  "options": { "num_ctx": 32768 }
}'

In Python with the ollama package:

import ollama
r = ollama.chat(model="qwen3:14b",
                messages=[{"role": "user", "content": "Hello"}],
                options={"num_ctx": 32768})

The OpenAI-compatible endpoint (/v1/chat/completions) does not accept num_ctx, because it is not part of the OpenAI format. Tools that use it, such as many coding agents and chat apps, get the server default. For them, use Method 1 or point the tool at a Modelfile model from Method 3.

Many apps also have their own setting: Open WebUI has “Context Length (Ollama)” under the model’s advanced parameters. More examples are in my Ollama API guide.

How Much Memory a Bigger Context Costs

Context is not free. Ollama reserves memory for the KV cache, which stores information about every token in the window. It grows in a straight line with the context length, and it is allocated when the model loads, whether you use the space or not.

The exact size depends on the model’s design (layers, attention heads, and whether it uses grouped-query or sliding-window attention). As a rough guide for a typical modern 7-14B model with the default f16 cache:

Context Approx. extra memory (f16 cache) With q8_0 cache With q4_0 cache
4k 0.25-0.7 GB about half about a quarter
16k 1-3 GB 0.5-1.5 GB 0.25-0.8 GB
32k 2-6 GB 1-3 GB 0.5-1.5 GB
128k 8-22 GB 4-11 GB 2-5.5 GB

These are ranges, not exact numbers. The real test is ollama ps: after you raise the context, check that SIZE still fits in your VRAM and PROCESSOR still says 100% GPU.

Also note: OLLAMA_NUM_PARALLEL multiplies the cost. Ollama’s FAQ explains that 4 parallel requests with a 2k context allocate an 8k context in total.

How to Fit More Context in VRAM

  1. Quantize the KV cache. Set OLLAMA_KV_CACHE_TYPE=q8_0. Ollama’s docs say it uses about half the memory of f16 with a very small loss in precision that usually has no noticeable effect. q4_0 uses about a quarter, with more loss at long contexts. It is a global setting and works when Flash Attention is active.
  2. Make sure Flash Attention is on. Ollama now uses it automatically where supported. You can force it with OLLAMA_FLASH_ATTENTION=1. It reduces memory use as the context grows.
  3. Use a smaller quant of the model. Going from Q8 to Q4_K_M frees several GB for context. See best Ollama models by VRAM.
  4. Keep OLLAMA_NUM_PARALLEL at 1 unless you really need parallel requests.
  5. Unload other models. Run ollama ps, then ollama stop <model> for anything you are not using.

Example for a 16 GB card running a 14B model at Q4_K_M with a 32k context:

OLLAMA_CONTEXT_LENGTH=32768
OLLAMA_KV_CACHE_TYPE=q8_0
OLLAMA_FLASH_ATTENTION=1

What Size to Choose

Use Suggested context
Short chats, quick questions 4k-8k
Long chats, summarising articles 16k-32k
Chat with PDFs, RAG 16k-32k (plus a good retriever)
Coding agents, web search, tool use 64k or more (Ollama’s own recommendation)
Whole codebases or books 128k+, only with enough VRAM

Bigger is not always better. A huge context makes the first response slower (the model has to read everything) and many models get less accurate with very long inputs. Use the smallest context that fits your task. For documents, my chat with PDFs locally guide shows how retrieval keeps the context small.

Troubleshooting

  • The setting does not change anything. On Windows and macOS, Ollama must be fully quit and restarted after changing an environment variable. Check the result with ollama ps.
  • The model got much slower after raising the context. Look at PROCESSOR in ollama ps. If part runs on the CPU, lower the context, quantize the cache, or use a smaller model. My Ollama not using GPU fix covers the rest.
  • Out of memory errors when loading. The context is too big for your VRAM and RAM. Halve it and try again.
  • My coding tool still forgets things. It probably uses the OpenAI-compatible endpoint. Set OLLAMA_CONTEXT_LENGTH or use a Modelfile model.
  • Answers get worse at very long contexts. Try q8_0 instead of q4_0 for the cache, or a smaller context.

FAQ

What is the maximum context length in Ollama?

It is the model’s own limit, shown by ollama show <model>. Ollama will not go above it. In practice, your memory is the real limit.

Is num_ctx the same as max tokens?

No. num_ctx is the whole window (input plus output). num_predict limits only how many tokens the model writes in its answer.

Does a bigger context slow down every reply?

Memory is reserved at load time, but speed mostly depends on how much text is actually in the window. Short chats stay fast even with a large context, as long as the model stays fully on the GPU.

Does LM Studio work the same way?

Similar idea, different place: LM Studio sets the context length in the model load settings. See how to use LM Studio.

Does the context reset between chats?

Yes. Each new conversation starts with an empty context. Ollama itself does not remember past chats unless your app sends the history again.

Sources

Free weekly email

Get new AI guides every week

One short weekly email with our newest local AI, ComfyUI and AI tool guides. No spam, no pop-ups, just the good stuff.

Free. Unsubscribe anytime in one click. We only use your email to send this newsletter. See our privacy policy.

Woman in a cosy home office holding a tablet showing the Ollama logo, hook 4K is too small, fix Ollama