Your local model forgets the start of a long chat, ignores half of the PDF you pasted, or a coding agent loses track of files after a few steps. With Ollama, the cause is very often the context length.
Ollama picks a default based on your VRAM, and on most home GPUs that default is only 4,096 tokens, far less than the model can handle.
This guide shows how to check your current context length, every way to change it (app, environment variable, CLI, Modelfile, API), how much extra memory each size costs, and how to fit a bigger context without spilling onto the CPU.
- Default: 4k tokens under 24 GB VRAM, 32k from 24-48 GB, 256k at 48 GB or more.
- Check it: run
ollama psand read theCONTEXTcolumn. - Change it for everything: set
OLLAMA_CONTEXT_LENGTH=32768and restart Ollama, or use the slider in the Ollama app settings. - Change it for one model:
PARAMETER num_ctx 32768in a Modelfile, or"options": {"num_ctx": 32768}in the API. - Agents and coding tools: Ollama recommends at least 64,000 tokens. Use
OLLAMA_KV_CACHE_TYPE=q8_0to roughly halve the memory cost.
New to Ollama? My Ollama guide covers install, models, commands and fixes in one place.
What Context Length Means
The context is everything the model can “see” at once: the system prompt, the chat history, any files or search results you paste in, and the answer it is writing. It is measured in tokens; one token is roughly three quarters of an English word.
When a conversation grows past the context length, Ollama has to drop the oldest part to make room. The model then simply does not know what was said earlier.
That is why a small context feels like the model has a short memory, and why it breaks tools that send long instructions, such as coding agents.
Two different numbers matter:
- The model’s maximum context: what it was trained for, for example 128k or 256k tokens. Run
ollama show <model>and look at the “context length” line. - The context Ollama actually allocates: what you set (or the default). This is what decides memory use and what the model can see.
Ollama’s Default Context Length
According to Ollama’s current documentation, the default depends on how much VRAM you have:
| Your VRAM | Default context | Typical GPUs |
|---|---|---|
| Under 24 GiB | 4k (4,096 tokens) | RTX 3060, 4060, 4070, 5070, 5080, most laptops |
| 24-48 GiB | 32k | RTX 3090, 4090, 5090, two 16 GB cards |
| 48 GiB or more | 256k | Workstation cards, large unified-memory Macs |
Cloud models on Ollama run at their maximum context by default. Older guides say the default is 2,048 tokens; that was true for early Ollama versions and is still listed in some parts of the docs, but the VRAM-based defaults above are what current versions use.
For most people on a 12-16 GB card, that means a 4k context out of the box. A 4k window is fine for short chats, but too small for pasting documents, RAG, or agents.
How to Check Your Current Context
Load a model (send it any message), then run:
ollama ps
You will see something like:
NAME ID SIZE PROCESSOR CONTEXT UNTIL
qwen3:14b a1b2c3d4e5f6 13 GB 100% GPU 32768 4 minutes from now
- CONTEXT is the context length Ollama allocated for this model.
- PROCESSOR shows where the model runs.
100% GPUis what you want. A split such as25%/75% CPU/GPUmeans part of it spilled into system RAM, which is much slower. A bigger context is the most common reason for that split.
Every Way to Change It
| Method | Applies to | Lasts | Best for |
|---|---|---|---|
| Ollama app slider | All models | Until changed | Desktop app users |
OLLAMA_CONTEXT_LENGTH |
All models (default) | Until changed | Servers, Docker, apps that cannot set options |
/set parameter num_ctx |
Current ollama run session |
Until you exit | Quick tests |
Modelfile PARAMETER num_ctx |
One custom model | Permanent | A model you always want with a big context |
API options.num_ctx |
One request | That request | Your own scripts and apps |
When more than one is set, the most specific wins: a request’s num_ctx beats the model’s Modelfile value, which beats the server default.
Method 1: OLLAMA_CONTEXT_LENGTH (All Models)
This changes the default for every model. It is the most reliable fix for tools that talk to Ollama but cannot send their own settings.
Ollama app: open Settings and move the context length slider. That is the same setting.
Windows:
- Quit Ollama from the system tray (right-click the icon, Quit).
- Open Start, search “environment variables”, and choose “Edit environment variables for your account”.
- Click New. Name:
OLLAMA_CONTEXT_LENGTH. Value:32768(or your choice). - Click OK, then start Ollama again from the Start menu.
macOS (app):
launchctl setenv OLLAMA_CONTEXT_LENGTH 32768
Then quit and reopen the Ollama app.
Linux (systemd service):
sudo systemctl edit ollama.service
Add these lines, save, then reload and restart:
[Service]
Environment="OLLAMA_CONTEXT_LENGTH=32768"
sudo systemctl daemon-reload
sudo systemctl restart ollama
One-off from a terminal:
OLLAMA_CONTEXT_LENGTH=32768 ollama serve
Docker: add -e OLLAMA_CONTEXT_LENGTH=32768 to your docker run command, or an environment: entry in Compose. See my Ollama Docker guide.
The same method sets every other Ollama variable. My Ollama commands cheat sheet lists them all.
Method 2: Inside ollama run
For a quick test, start a chat and type:
ollama run qwen3:14b
>>> /set parameter num_ctx 32768
The model reloads with the new context. It only lasts for this session. To keep it, save the session as a new model:
>>> /save qwen3-14b-32k
Next time, run ollama run qwen3-14b-32k.
Method 3: A Modelfile (One Model, Permanent)
This is the cleanest way to have one model always load with a large context, while other models keep the default. Create a text file named Modelfile:
FROM qwen3:14b
PARAMETER num_ctx 32768
Then build the new model:
ollama create qwen3-14b-32k -f Modelfile
This does not copy the weights; it reuses the existing model and only stores the new settings, so it takes seconds and almost no disk space. Use the new name in any app. My Ollama Modelfile guide covers every other parameter you can set the same way.
Method 4: The API and Apps
With Ollama’s native API, send num_ctx in options:
curl http://localhost:11434/api/chat -d '{
"model": "qwen3:14b",
"messages": [{"role": "user", "content": "Summarise this report..."}],
"options": { "num_ctx": 32768 }
}'
In Python with the ollama package:
import ollama
r = ollama.chat(model="qwen3:14b",
messages=[{"role": "user", "content": "Hello"}],
options={"num_ctx": 32768})
The OpenAI-compatible endpoint (/v1/chat/completions) does not accept num_ctx, because it is not part of the OpenAI format. Tools that use it, such as many coding agents and chat apps, get the server default. For them, use Method 1 or point the tool at a Modelfile model from Method 3.
Many apps also have their own setting: Open WebUI has “Context Length (Ollama)” under the model’s advanced parameters. More examples are in my Ollama API guide.
How Much Memory a Bigger Context Costs
Context is not free. Ollama reserves memory for the KV cache, which stores information about every token in the window. It grows in a straight line with the context length, and it is allocated when the model loads, whether you use the space or not.
The exact size depends on the model’s design (layers, attention heads, and whether it uses grouped-query or sliding-window attention). As a rough guide for a typical modern 7-14B model with the default f16 cache:
| Context | Approx. extra memory (f16 cache) | With q8_0 cache | With q4_0 cache |
|---|---|---|---|
| 4k | 0.25-0.7 GB | about half | about a quarter |
| 16k | 1-3 GB | 0.5-1.5 GB | 0.25-0.8 GB |
| 32k | 2-6 GB | 1-3 GB | 0.5-1.5 GB |
| 128k | 8-22 GB | 4-11 GB | 2-5.5 GB |
These are ranges, not exact numbers. The real test is ollama ps: after you raise the context, check that SIZE still fits in your VRAM and PROCESSOR still says 100% GPU.
Also note: OLLAMA_NUM_PARALLEL multiplies the cost. Ollama’s FAQ explains that 4 parallel requests with a 2k context allocate an 8k context in total.
How to Fit More Context in VRAM
- Quantize the KV cache. Set
OLLAMA_KV_CACHE_TYPE=q8_0. Ollama’s docs say it uses about half the memory of f16 with a very small loss in precision that usually has no noticeable effect.q4_0uses about a quarter, with more loss at long contexts. It is a global setting and works when Flash Attention is active. - Make sure Flash Attention is on. Ollama now uses it automatically where supported. You can force it with
OLLAMA_FLASH_ATTENTION=1. It reduces memory use as the context grows. - Use a smaller quant of the model. Going from Q8 to Q4_K_M frees several GB for context. See best Ollama models by VRAM.
- Keep
OLLAMA_NUM_PARALLELat 1 unless you really need parallel requests. - Unload other models. Run
ollama ps, thenollama stop <model>for anything you are not using.
Example for a 16 GB card running a 14B model at Q4_K_M with a 32k context:
OLLAMA_CONTEXT_LENGTH=32768
OLLAMA_KV_CACHE_TYPE=q8_0
OLLAMA_FLASH_ATTENTION=1
What Size to Choose
| Use | Suggested context |
|---|---|
| Short chats, quick questions | 4k-8k |
| Long chats, summarising articles | 16k-32k |
| Chat with PDFs, RAG | 16k-32k (plus a good retriever) |
| Coding agents, web search, tool use | 64k or more (Ollama’s own recommendation) |
| Whole codebases or books | 128k+, only with enough VRAM |
Bigger is not always better. A huge context makes the first response slower (the model has to read everything) and many models get less accurate with very long inputs. Use the smallest context that fits your task. For documents, my chat with PDFs locally guide shows how retrieval keeps the context small.
Troubleshooting
- The setting does not change anything. On Windows and macOS, Ollama must be fully quit and restarted after changing an environment variable. Check the result with
ollama ps. - The model got much slower after raising the context. Look at
PROCESSORinollama ps. If part runs on the CPU, lower the context, quantize the cache, or use a smaller model. My Ollama not using GPU fix covers the rest. - Out of memory errors when loading. The context is too big for your VRAM and RAM. Halve it and try again.
- My coding tool still forgets things. It probably uses the OpenAI-compatible endpoint. Set
OLLAMA_CONTEXT_LENGTHor use a Modelfile model. - Answers get worse at very long contexts. Try
q8_0instead ofq4_0for the cache, or a smaller context.
FAQ
What is the maximum context length in Ollama?
It is the model’s own limit, shown by ollama show <model>. Ollama will not go above it. In practice, your memory is the real limit.
Is num_ctx the same as max tokens?
No. num_ctx is the whole window (input plus output). num_predict limits only how many tokens the model writes in its answer.
Does a bigger context slow down every reply?
Memory is reserved at load time, but speed mostly depends on how much text is actually in the window. Short chats stay fast even with a large context, as long as the model stays fully on the GPU.
Does LM Studio work the same way?
Similar idea, different place: LM Studio sets the context length in the model load settings. See how to use LM Studio.
Does the context reset between chats?
Yes. Each new conversation starts with an empty context. Ollama itself does not remember past chats unless your app sends the history again.
Sources
- Ollama docs: Context length (VRAM-based defaults, 64k recommendation, ollama ps)
- Ollama FAQ (OLLAMA_CONTEXT_LENGTH, num_ctx, KV cache types, parallel requests)
- Ollama source: envconfig/config.go



