KoboldCpp is a free, open-source app that runs AI language models on your own PC from a single file. No installer, no Python, no account: download one program, pick a GGUF model, click Launch, and chat in your browser.
It is built on llama.cpp and goes far beyond chat. It can also generate images and video, transcribe speech, read text aloud, make music, connect to MCP tools, and act as a drop-in local API for apps that expect OpenAI, Ollama or KoboldAI.
This guide covers how to download the right file, choose a model, set GPU layers and context, use the web UI and API, and fix common errors. Everything is based on the official KoboldCpp GitHub README, checked in November 2026.
- Download only from GitHub (LostRuins/koboldcpp releases). The README warns that koboldcpp.com is a fake phishing site.
- Windows:
koboldcpp.exe(NVIDIA CUDA),koboldcpp-nocuda.exe(CPU and Vulkan),koboldcpp-oldpc.exe(older hardware). - Mac:
koboldcpp-mac-arm64for Apple Silicon. Linux:koboldcpp-linux-x64. - Models are separate: download a GGUF from Hugging Face. The README’s top pick is Qwen3-VL-8B (Q4_K_S); Gemma 3 4B for light PCs.
- Launch: choose the model in GGUF Text Model, click Launch, open
http://localhost:5001. - Speed: use CUDA (NVIDIA) or Vulkan (AMD/Intel/NVIDIA). GPU layers
-1= autofit. - Gibberish replies? Relaunch with
--jinjato use the model’s own chat template.
What Is KoboldCpp?
KoboldCpp is inspired by KoboldAI and built on llama.cpp. It ships as one portable executable for Windows, Linux and macOS, runs on CPU or GPU, and supports full or partial GPU offloading.
| Feature | What it can do |
|---|---|
| Text | Any GGUF model in chat, instruct, story or adventure modes; vision models can read images |
| Images | Stable Diffusion 1.5, SDXL, SD3, Flux, Qwen Image, Z-Image and more |
| Video | WAN 2.2, LTX 2.3, MiniMax H3 and more |
| Voice in | Whisper speech-to-text |
| Voice out | Kokoro, Qwen3TTS, OuteTTS, Parler, Dia |
| Music | ACE Step 1.5 and ACE Step XL |
| Tools | MCP servers, tool calling, web search, RAG via TextDB, and a built-in KoboldCpp Agent |
| Writing tools | KoboldAI Lite UI with memory, world info, author’s notes and saved stories |
KoboldCpp vs Ollama vs LM Studio
| KoboldCpp | Ollama | LM Studio | |
|---|---|---|---|
| Install | None, single file | Installer | Installer |
| Open source | Yes | Yes | No (free) |
| Model download | You bring GGUF files | Built-in library | Built-in search |
| Images, video, TTS, music | Yes, built in | No (text and vision) | No (text and vision) |
| Story writing tools | Excellent | None | None |
| Default port | 5001 | 11434 | 1234 |
| Best for | Writers, tinkerers, all-in-one local AI | Developers, simple CLI | Beginners who like a polished app |
Prefer a classic desktop app? Read our LM Studio beginner guide. Want the engine underneath? See our llama.cpp guide.
Step 1: Download the Right File
Safety first: the KoboldCpp README warns that koboldcpp.com is a malicious fake site. Only download from the official GitHub releases page (github.com/LostRuins/koboldcpp/releases/latest). The official website is koboldcpp.net.
| Your system | File to download |
|---|---|
| Windows + NVIDIA GPU | koboldcpp.exe |
| Windows + AMD/Intel GPU or CPU only | koboldcpp-nocuda.exe (smaller, CPU and Vulkan) |
| Old Windows PC that crashes | koboldcpp-oldpc.exe |
| Linux x86-64 | koboldcpp-linux-x64 (or -nocuda / -oldpc) |
| Mac with Apple Silicon | koboldcpp-mac-arm64 |
| Intel Mac, Raspberry Pi, Android | Build from source (Android via Termux) |
- Windows: double-click the .exe to open the launcher.
- Linux:
chmod +x koboldcpp-linux-x64then./koboldcpp-linux-x64 - Mac:
chmod +x koboldcpp-mac-arm64then run it from Terminal. If macOS blocks it, allow it in System Settings > Privacy & Security.
Step 2: Get a GGUF Model
KoboldCpp does not include models. Download a GGUF file from Hugging Face (Bartowski’s collection is a popular source). The README suggests:
| Use | Suggested model | Notes |
|---|---|---|
| Best all-rounder | Qwen3-VL-8B Instruct (Q4_K_S) | Add its mmproj file for image understanding |
| Light and fast | Gemma 3 4B (Q4_K_M) | Good on 8 GB RAM PCs; mmproj available for vision |
| Images | PicX Real (Q8_0 GGUF) | Or other models in the koboldcpp/imgmodel repo |
| Speech-to-text | Whisper Base (English) | Bigger Whisper models are more accurate |
| Text-to-speech | Kokoro TTS | Natural narration voices |
Unsure which file to pick? Our Hugging Face download guide explains quant names like Q4_K_M, and our VRAM guide shows what fits your GPU. The launcher’s Get Help > Newbie Templates also gives ready-made setups.
Step 3: Launch and Pick Settings
- Open KoboldCpp and select your file in the GGUF Text Model field.
- Pick a backend: CUDA for NVIDIA, Vulkan for AMD, Intel or NVIDIA, or CPU.
- GPU layers: leave at
-1(autofit) to start. Use0to stay on CPU. - Context size: 4K to 8K tokens is a good start. Bigger context uses more memory.
- Click Launch and wait until loading finishes. Keep the window open.
- Open
http://localhost:5001in your browser. - Save your setup as a launch config so next time it is one click.
Step 4: Use the Web UI
The bundled KoboldAI Lite UI has several modes:
- Instruct: ChatGPT-style question and answer. Best for most people.
- Chat: a back-and-forth conversation with a named character, including a messenger-style view.
- Story: co-write fiction; the model continues your text.
- Adventure: text-adventure style where you type actions.
Useful writing tools include Memory (facts always sent to the model), World Info (details triggered by keywords), Author’s Note (style guidance) and saved stories. A llama.cpp-style UI is also available at /lcpp.
Images, Voice, Music and Vision
- Vision: load the matching mmproj file under Loaded Files > Mmproj File, or add
--mmproj. - Images: load an image model and open the StableUI at
/sdui. - Music: load an ACE Step model and open
/musicui. - Voice: load Whisper for speech input and a TTS model for spoken replies.
- Agent: select Launch KoboldCpp Agent in the Admin tab, or add
--agent. It can write and edit code, run programs and schedule tasks, so only enable it if you understand the risk.
Each extra model uses more memory, so load only what you need at once.
Use KoboldCpp as a Local API
| Interface | URL |
|---|---|
| KoboldCpp API | http://localhost:5001 |
| OpenAI-compatible | http://localhost:5001/v1 |
| Interactive API docs | http://localhost:5001/api |
| Image UI | http://localhost:5001/sdui |
| Music UI | http://localhost:5001/musicui |
It also speaks the Anthropic, Ollama, AUTOMATIC1111/Forge, ComfyUI and XTTS APIs, so many apps can connect by changing only the base URL. Point an app that expects Ollama at port 5001 and it often just works.
Command-Line Launch and Key Flags
koboldcpp.exe --model "C:\Models\model.gguf" --usecuda --gpulayers -1 --contextsize 8192
| Flag | What it does |
|---|---|
--model |
Path to the GGUF text model |
--usecuda / --usevulkan |
GPU backend (CUDA is NVIDIA only) |
--gpulayers N |
-1 autofit, 0 CPU only, or a number of layers |
--contextsize N |
Maximum tokens the model can work with |
--batchsize N |
Prompt batch size; smaller uses less memory |
--mmproj |
Vision projector file |
--jinja |
Use the model’s built-in chat template |
--port |
Change from 5001 |
--noavx2 |
For older CPUs that crash |
--nomodel |
Start the UI with no local model (use an external provider) |
--debugmode |
Extra logging for load errors |
Run with --help to see every option.
No GPU? Colab, RunPod and External Providers
- Google Colab: the official KoboldCpp Colab notebook runs it on a free cloud GPU in a couple of minutes (follow Colab’s usage rules).
- RunPod: a ready KoboldCpp image for rented GPUs.
- External providers: start with
--nomodel(or Allow Launch Without Models) and connect the UI to OpenAI-compatible APIs, Anthropic, OpenRouter, Gemini, Grok, Mistral or AI Horde. - Docker: an official image exists, but the README says it is for experts and cloud GPU rentals; use the normal binaries at home.
Troubleshooting
| Problem | Fix |
|---|---|
| Out of RAM | Smaller model or shorter context; lower --batchsize |
| Out of VRAM | Fewer GPU layers, smaller model or shorter context (autofit is only an estimate) |
| Very slow | Make sure CUDA or Vulkan is selected and layers are offloaded; use the CUDA .exe on NVIDIA |
| Crashes at startup on old PC | Use the oldpc build or --noavx2 |
| Browser can’t connect | Wait for loading to finish; check the address in the terminal (default port 5001) |
| Model won’t load | Read the terminal error, check free memory, update KoboldCpp, try --debugmode |
| Incoherent or rambling replies | Wrong chat template: relaunch with --jinja or turn on the Jinja toggle |
| Mac says the app is blocked | Allow it in System Settings > Privacy & Security |
| Downloaded from a strange site | Delete it and download only from GitHub releases |
My Take
KoboldCpp is the Swiss Army knife of local AI. One file gives you chat, story writing, images, voice and music, plus an API that pretends to be OpenAI, Ollama or A1111 for other apps.
It is less hand-holding than LM Studio because you fetch models yourself, but once you have one GGUF downloaded it takes a minute to run. For writers and tinkerers it is hard to beat, just remember to download it only from GitHub.
FAQ
Is KoboldCpp free?
Yes. It is free and open source, and runs fully on your own computer.
Is KoboldCpp safe?
The official GitHub release is. The project warns that koboldcpp.com is a fake site, so only download from github.com/LostRuins/koboldcpp.
Does KoboldCpp need a GPU?
No. It runs on CPU, but a GPU with CUDA or Vulkan makes it much faster.
What port does KoboldCpp use?
Port 5001 by default. The OpenAI-compatible API is at http://localhost:5001/v1.
Which model should I start with?
The README’s top pick is Qwen3-VL-8B at Q4_K_S. On a low-memory PC, try Gemma 3 4B.
Can KoboldCpp generate images?
Yes. Load a supported image model such as SDXL or Flux and use the image UI at /sdui.
Can I use KoboldCpp with apps made for Ollama or OpenAI?
Yes. It offers OpenAI and Ollama-compatible endpoints on port 5001.




