Ollama Guide (2026): Install, Best Models, Commands, API and Fixes

Ollama is the easiest way to run large language models on your own computer: one installer, one command to download a model, and a local API that most AI apps can talk to. It is also easy to get wrong. The default context is small, models silently fall back to the CPU, and the models folder grows until your drive is full. I have written a separate deep guide for each of those jobs. This page puts them in order, gives you the short answer for each step, and links to the full walkthrough when you need it.

What Is Ollama?

Ollama is a free, open-source program that downloads, runs and serves local LLMs. Under the hood it uses llama.cpp to run GGUF models on NVIDIA, AMD and Apple GPUs (or the CPU), and it wraps that engine in a simple command line, a background service and an HTTP API on port 11434. You type ollama run plus a model name, and Ollama pulls the right file, loads it onto your GPU and opens a chat.

Why people use it instead of a cloud chatbot: it is private (prompts never leave your machine), it costs nothing per token, it works offline, and it plugs into coding assistants, chat front-ends and automation tools that expect an OpenAI-style API.

What it is not: Ollama does not make a model smarter than its size allows, and model tags ending in -cloud run on Ollama’s servers, not on your PC.

The Ollama Roadmap: Which Guide Do You Need?

Step 1: Install Ollama

• Windows 10 22H2 or newer: run irm https://ollama.com/install.ps1 | iex in PowerShell, or download OllamaSetup.exe. No admin rights are needed. NVIDIA users need driver 551.61 or newer.
• macOS 14 Sonoma or newer: download the app and drag it into Applications. Apple silicon Macs use the GPU; Intel Macs run on the CPU only.
• Linux: curl -fsSL https://ollama.com/install.sh | sh. It installs a systemd service that starts on boot.
• Docker: the official image works with NVIDIA and AMD GPUs once the container toolkit is set up. See Ollama in Docker, which also includes an Open WebUI compose file.

Full steps, screenshots of what to expect and the GPU checks are in How to Install Ollama on Windows, Mac and Linux.

Step 2: Check Your Hardware

The rule that decides everything: the model file plus its context (KV cache) should fit in your GPU’s VRAM. If it does not, Ollama moves part of the model to system RAM, and speed drops sharply. A rough guide:

System RAM matters too, especially for big mixture-of-experts models that run partly on the CPU. See how much RAM you need for local AI. Intel Arc owners should read Intel Arc for local AI.

Step 3: Pick a Model

Short version from my best Ollama models list:

• 8 GB GPU: mimo-v2.6:9b for coding and agents, gemma4:12b for chat and vision.
• 12-16 GB GPU: gemma4:12b or granite4.2:8b fully on the GPU; qwen3.8:27b with a little CPU offload is the smartest model most people can run.
• 24 GB GPU: qwen3.8:27b as the all-rounder, gemma4:31b for reasoning and maths.
• 64-128 GB RAM: big MoE models such as gpt-oss:120b run with CPU offload.

Any GGUF on Hugging Face also works directly: ollama run hf.co/USER/REPO-GGUF:Q4_K_M. If you are unsure which quant to pick, start with Q4_K_M; the GGUF guide explains the rest. Looking for a specific model family? I have walkthroughs for running DeepSeek locally and many others in the Local LLM category.

Step 4: Learn the Commands

You only need eight commands day to day:

Slash commands inside the chat, Modelfile options, every environment variable and the API endpoints are in the Ollama commands cheat sheet.

Step 5: Settings That Matter

• Context length. Ollama’s default context is small on most GPUs, which is why long chats and documents “forget” things. Set OLLAMA_CONTEXT_LENGTH=32768 (or whatever fits your VRAM) as an environment variable, or /set parameter num_ctx 32768 inside a chat.
• KV cache quantization. OLLAMA_KV_CACHE_TYPE=q8_0 roughly halves the memory used by the context, with flash attention (automatic on supported GPUs).
• Keep models loaded. OLLAMA_KEEP_ALIVE controls how long a model stays in memory after the last request.
• Listen on your network. OLLAMA_HOST=0.0.0.0 lets other devices reach the API. Only do this on a trusted network.
• Models folder. OLLAMA_MODELS moves storage to another drive.

How to set these on each OS (Windows environment variables, launchctl on macOS, systemd overrides on Linux) is covered in the cheat sheet.

Step 6: Add a Chat Interface or Connect Apps

• Web chat: Open WebUI gives you a ChatGPT-style interface with history, documents and multiple users.
• Your own code: the Ollama API guide shows Python, curl and OpenAI-SDK examples against http://localhost:11434.
• Coding assistants: VS Code extensions such as Cline and Continue can point at Ollama. You can even route Claude Code to a local model with the Claude Code Router.

Step 7: Manage Storage

Models are big (a 27B model at Q4 is about 17 GB), and they add up fast. Default folders: Windows C:\Users\<you>\.ollama\models, macOS ~/.ollama/models, Linux service /usr/share/ollama/.ollama/models.

• Delete one model with ollama rm <name>, or all of them with the one-liners in delete Ollama models and uninstall Ollama.
• Move the whole library to a bigger drive with OLLAMA_MODELS: move Ollama models to another drive.
• The Windows uninstaller does not remove moved models or %USERPROFILE%\.ollama; delete those by hand for a full clean-up.

Ollama vs LM Studio, llama.cpp and vLLM

Pick Ollama for one-command setup and a stable API that apps support. Pick LM Studio for a desktop app with a model browser and sliders, especially on a Mac (MLX models). Pick llama.cpp for full control over offload, multi-GPU splits and the newest models first. On the same GGUF and GPU, the three are usually within about 10% of each other, because Ollama and LM Studio use llama.cpp for GGUF. Settings matter more than the tool. Full comparison: Ollama vs LM Studio vs llama.cpp. For serving many users at once, see vLLM vs llama.cpp vs Ollama. LM Studio users with load errors can check LM Studio not loading models.

Common Problems and Fixes

Frequently Asked Questions

Is Ollama free?

Yes. Ollama is free and open source, and local models cost nothing to run beyond electricity. Only the optional -cloud models use Ollama’s paid servers.

Does Ollama need a GPU?

No, but it is much faster with one. Small models (1-4B) run acceptably on a modern CPU; anything bigger really wants an NVIDIA, AMD or Apple silicon GPU.

Is Ollama private?

Local models run entirely on your machine, so prompts and files stay on it. Cloud model tags send requests to Ollama’s servers.

Where does Ollama store models?

In .ollama/models in your user folder on Windows and macOS, and /usr/share/ollama/.ollama/models for the Linux service, unless OLLAMA_MODELS points elsewhere.

Which model should I start with?

gemma4:e4b on 8 GB cards and qwen3.8:27b on 24 GB cards are safe first picks. See the best Ollama models list for more.

Can Ollama run Hugging Face GGUF files?

Yes: ollama run hf.co/USER/REPO-GGUF:Q4_K_M, or import a local file with a Modelfile.

Final Thoughts

Install it, run one model that fits your VRAM, check ollama ps says GPU, then raise the context to what you actually need. That covers most of what makes Ollama feel fast and reliable. When you want more, add Open WebUI for a proper interface or the API for your own tools. I keep this page updated as I add Ollama guides, so bookmark it.

Sources

• Ollama downloads and github.com/ollama/ollama
• Ollama FAQ (environment variables, storage, context)
• Ollama API reference