AnythingLLM is a free, open-source app that turns your computer into a private ChatGPT that can read your own documents. You drop in PDFs, Word files or web pages, pick a local or cloud AI model, and ask questions.
Answers come back with citations to the exact source, and with a local model nothing leaves your PC.
It also does a lot more than document chat: AI agents that browse the web and save files, MCP tools, scheduled tasks, memories, a developer API and, in the Docker version, multiple users and an embeddable chat widget for your website.
This guide walks you through installing the desktop app or Docker version, setting up your first workspace, connecting Ollama or LM Studio, getting accurate answers from documents, using agents, and fixing common problems. Details come from the official AnythingLLM docs and GitHub repo (v1.20.0), checked in November 2026.
- What: free, open-source “chat with your docs” app with agents, by Mintplex Labs. Current version 1.20.0.
- Desktop: download from anythingllm.com/download for Windows, Mac or Linux. Single user, includes a built-in local LLM.
- Docker:
docker pull mintplexlabs/anythingllm, run on port 3001, openhttp://localhost:3001. Adds multi-user, passwords and a chat widget. - Recommended PC: 16 GB RAM and an 8-core CPU; a GPU with 8-12 GB VRAM on Windows for local models. Any M-series Mac works.
- Agents: type
@agentin a workspace,/exitto stop. - Privacy: turn off anonymous telemetry in the sidebar under Privacy, or set
DISABLE_TELEMETRY=true.
What Is AnythingLLM?
AnythingLLM is an all-in-one AI application from Mintplex Labs. You connect any large language model, feed it your documents, and chat with them.
Under the hood it handles the hard parts of retrieval-augmented generation (RAG): splitting documents into chunks, turning them into embeddings, storing them in a vector database and pulling the right pieces into each answer.
Out of the box it uses its own native embedder and a built-in LanceDB vector database, so there is nothing extra to install. It reads PDF, TXT, DOCX and other common file types, and can also scrape websites.
| Feature | What it means for you |
|---|---|
| Workspaces | Separate “rooms” with their own documents, settings and chat history |
| Citations | Every answer shows which document chunks it used |
| Built-in LLM engine | Run local models without installing Ollama (powered by Lemonade and llama.cpp) |
| Agents and MCP | Web search, scraping, saving files, charts, SQL and any MCP server |
| Memories | Automatic and user-managed memories about you or a workspace |
| Scheduled tasks | Run prompts on a schedule with full agent abilities |
| Model routing | Send chats to different providers based on rules you set |
| Developer API | Use your workspaces from scripts and other apps |
Desktop vs Docker
| Desktop app | Docker | |
|---|---|---|
| Install | One-click installer | Docker command or Compose file |
| Users | Single user, no login | Multi-user with passwords and permissions |
| Built-in local LLM | On by default | Optional (ENABLE_BUILT_IN_LLM=true) |
| Access | App window on your PC | Any browser at port 3001 |
| Website chat widget | No | Yes |
| White-labelling and themes for all users | No | Yes |
| Documents, agents, API, any LLM | Yes | Yes |
Pick Desktop if it is just you on one computer. Pick Docker if you want a family or team server, browser access from other devices or a chatbot on your website.
System Requirements
| Item | Recommended (official) |
|---|---|
| RAM | 16 GB |
| CPU | Any 8-core CPU |
| Storage | Varies with the size of the local models you download |
| GPU (Windows) | Recommended for local models; 8-12 GB+ VRAM is great |
| Mac | Any M-series chip handles local models; Intel Macs are slower |
| Docker server | At least 2 GB RAM and 10 GB disk (when using a remote LLM) |
If you only use cloud models such as OpenAI, Claude or Gemini, the requirements are much lower because the app itself is light. A PC without a GPU can also connect to a model running on another machine on your network.
Install the Desktop App
- Go to anythingllm.com/download and pick Windows, Mac or Linux.
- Run the installer. On Windows you may need to click More info then Run anyway if SmartScreen warns you.
- Open AnythingLLM. The onboarding screens ask you to choose an LLM. The built-in provider is the default and runs models locally.
- Pick a model to download (start with a small one, around 3-8B parameters, if you have 8 GB of VRAM or less).
- Keep the default embedder and vector database (AnythingLLM Native Embedder and LanceDB). They are private and need no setup.
- Create your first workspace when asked. You are ready to chat.
Note that the desktop app has no accounts or password. The docs warn against exposing it to the internet. Its “network discovery” option (off by default) only lets AnythingLLM Mobile, scripts or the API reach it on your local network.
Install With Docker
The Docker image supports both amd64 and arm64. The official guide mounts a folder on your PC so your data survives updates.
- Install Docker Desktop (Windows/Mac) or Docker Engine (Linux).
- Pull the image:
docker pull mintplexlabs/anythingllm - On Windows, open PowerShell and run the official command, which creates
Documents\anythingllmand a.envfile, then starts the container:$env:STORAGE_LOCATION="$HOME\Documents\anythingllm"; If(!(Test-Path $env:STORAGE_LOCATION)) {New-Item $env:STORAGE_LOCATION -ItemType Directory}; If(!(Test-Path "$env:STORAGE_LOCATION\.env")) {New-Item "$env:STORAGE_LOCATION\.env" -ItemType File}; docker run -d --rm -p 3001:3001 --cap-add SYS_ADMIN -v "$env:STORAGE_LOCATION`:/app/server/storage" -v "$env:STORAGE_LOCATION\.env:/app/server/.env" -e STORAGE_DIR="/app/server/storage" mintplexlabs/anythingllm - On Linux or Mac, use the same idea with
export STORAGE_LOCATION=$HOME/anythingllm,mkdir -pandtouchfor the .env file, then the samedocker runflags. - Open
http://localhost:3001and finish onboarding. - For a shared server, turn on multi-user mode in settings and create accounts.
Important: inside Docker, localhost means the container, not your PC. To reach Ollama or LM Studio running on the host, use http://host.docker.internal:11434 (Ollama) or http://host.docker.internal:1234 (LM Studio). On Linux add --add-host=host.docker.internal:host-gateway to the run command.
To use the built-in engine in Docker, set ENABLE_BUILT_IN_LLM=true. On first boot it downloads the engine (a few MB) to engines/embedded-lemonade in your storage folder.
Choose Your AI Model
Change the model any time in Settings > AI Providers > LLM. You can also set a different model per workspace.
| Option | Best for | Setup |
|---|---|---|
| Built-in (Lemonade + llama.cpp) | Easiest local setup | Pick and download a model in the app |
| Ollama | If you already use Ollama | Base URL http://127.0.0.1:11434, choose a pulled model |
| LM Studio | Browsing GGUF models with a GUI | Start LM Studio’s server, base URL http://127.0.0.1:1234/v1 |
| KoboldCpp, LocalAI, text-generation-webui | Other local back ends | Point at their API URL |
| OpenAI, Anthropic, Gemini, xAI, DeepSeek, Mistral, Groq, OpenRouter and more | Best quality, no GPU needed | Paste your API key |
New to local models? Our Ollama guide and LM Studio guide show how to download and run them, and KoboldCpp is another light option.
For embeddings, the native embedder is fine for most people. If you already run Ollama, nomic-embed-text is a popular alternative (it is the example used in the official Docker Compose file).
Create a Workspace and Add Documents
- Click New Workspace in the sidebar and give it a name, such as “Tax 2026” or “Product manuals”.
- Click the upload icon next to the workspace name.
- Drag in your files, or paste a URL to fetch a web page.
- Select the uploaded files and click Move to Workspace.
- Click Save and Embed. Wait while AnythingLLM chunks and embeds them.
- Ask a question in the chat. Expand the citations under the answer to see the source text.
You can also drag files straight into the chat box. Pin an important document (the pin icon in the document list) to send its full text with every message instead of only the best-matching chunks, which helps with short but critical files.
Get Better Answers From Your Documents
- Use Query mode for strict answers. In the workspace’s chat settings, set the chat mode to Query so it only answers from your documents and says when it cannot find the answer. Chat mode mixes documents with the model’s general knowledge.
- Keep workspaces focused. One topic per workspace gives the retriever less noise.
- Raise the context snippets in the workspace’s vector database settings if answers miss details spread over many pages.
- Adjust the similarity threshold if it returns “no relevant information” too often (lower it) or pulls in junk (raise it).
- Use a bigger model. Small 1-3B models summarise poorly. 7-14B local models or a cloud model give much better answers.
- Clean scanned PDFs. Image-only PDFs have no text to embed. Run OCR first.
- Write a system prompt. Tell it who it is and how to answer, for example “Answer in bullet points and quote the page.”
For a side-by-side setup with Open WebUI as well, see our guide to chatting with PDFs locally.
Use AI Agents
Agents let the model use tools. Set them up once in the workspace’s Agent Configuration (choose an agent model and enable skills), then type @agent in the chat followed by your request. A popup shows which tools are on. Type /exit to end the agent session.
| Default skill | What it does |
|---|---|
| RAG Search | Looks up what the workspace documents say and can save summaries to agent memory |
| Web Browsing | Searches the internet for current information |
| Web Scraping | Reads a site and embeds it into the workspace |
| Save Files | Writes output to a file on your PC (you pick the location) |
| List and Summarize Documents | Shows or summarises workspace files |
| Chart Generation | Makes charts from a prompt or a CSV |
| SQL Agent | Reads databases and runs SELECT queries (use a read-only user) |
Example prompts:
@agent search the web for this week's RTX 5070 prices and save a summary to a file@agent scrape https://example.com/docs and summarise the setup steps@agent make a bar chart of monthly sales from sales.csv
The docs note that tool calling depends heavily on the model. Small 4-bit models often fail as agents, while the same model at 8-bit usually does better. If the agent ignores tools, try a higher quantisation, a larger model or a cloud model.
Beyond the built-in skills you can add MCP servers, build no-code Agent Flows and write custom agent skills.
Other Useful Features
- Memories: automatic and manual memories about you or a workspace.
- Scheduled tasks: run a prompt on a cron schedule, for example a morning news summary.
- Dynamic model routing: send coding questions to one model and quick chats to a cheaper one.
- Speech: built-in Whisper transcription for audio and video files, plus text-to-speech with the browser voice, local Piper, OpenAI or ElevenLabs.
- Browser extension to send pages to a workspace, and an open-source Android app.
- Developer API: create an API key in settings and query workspaces from your own code.
Privacy and Where Data Is Stored
| OS | Desktop storage folder |
|---|---|
| Windows | C:\Users\<user>\AppData\Roaming\anythingllm-desktop\storage |
| Mac | /Users/<user>/Library/Application Support/anythingllm-desktop/storage |
| Linux | ~/.config/anythingllm-desktop/storage/ |
You can also open it from the app by clicking the version number at the top right. Back up this folder to keep your workspaces, documents and chats.
With a local LLM, the native embedder and LanceDB, your documents never leave your computer. If you pick a cloud model, the relevant chunks of your documents are sent to that provider with each question.
AnythingLLM collects anonymous usage telemetry by default. Turn it off in the sidebar under Privacy, or set DISABLE_TELEMETRY="true" in the server or Docker .env file.
Troubleshooting
| Problem | Likely cause | Fix |
|---|---|---|
| Docker cannot connect to Ollama or LM Studio | localhost points inside the container |
Use http://host.docker.internal:11434; on Linux add the host-gateway flag |
| Ollama models not listed | Ollama not running or wrong URL | Run ollama list, check the base URL http://127.0.0.1:11434 |
| “No relevant information” answers | Documents not embedded or threshold too high | Click Save and Embed; lower the similarity threshold |
| Answers ignore my documents | Chat mode mixes general knowledge | Switch the workspace to Query mode, pin key files |
| PDF uploads but gives empty answers | Scanned image PDF | OCR the file first |
| Agent does not use tools | Model too small or low quant | Use an 8-bit or larger model, or a cloud model |
| Very slow replies | Model too big for VRAM, running on CPU | Pick a smaller model or a lower quant |
| Docker data lost after update | No storage volume mounted | Use the official command with STORAGE_LOCATION mounted |
| Browsing to the desktop port shows 404 | Desktop has no web UI | Use the app window, or Docker for browser access |
AnythingLLM vs Other Local AI Apps
| App | Strength | Pick it if |
|---|---|---|
| AnythingLLM | Document chat, agents, multi-user server | You mainly want to chat with files |
| LM Studio | Model browser and fast local runtime | You want to try many models |
| Jan | Simple offline ChatGPT-style app | You want a clean local chat app |
| Open WebUI | Feature-rich web UI for Ollama | You run Ollama on a server |
See our Jan AI guide for the simplest offline chat option.
My Take
AnythingLLM is the easiest way I know to get private “chat with your documents” working. The desktop app with the built-in model takes minutes, citations make it trustworthy, and Query mode stops it inventing answers. For a home or office server, the Docker version with multi-user mode is excellent.
Start with one focused workspace and a 7-8B model. If answers are weak, try a larger local model or plug in a cloud API key for the hard questions.
FAQ
Is AnythingLLM free?
Yes. The desktop app and the Docker version are free and open source. Mintplex Labs also offers a paid hosted instance.
Does AnythingLLM work offline?
Yes. With the built-in LLM (or Ollama/LM Studio), the native embedder and LanceDB, everything runs on your computer.
Do I need Ollama for AnythingLLM?
No. The desktop app includes a built-in local LLM engine. Ollama is optional.
What file types can AnythingLLM read?
PDF, TXT, DOCX and many other document types, plus web pages, and audio or video through built-in transcription.
How much RAM does AnythingLLM need?
The docs recommend 16 GB RAM and an 8-core CPU. With only cloud models it needs much less.
How do I use agents in AnythingLLM?
Configure an agent model in the workspace, then type @agent and your request in the chat. Type /exit to stop.
Can several people use one AnythingLLM?
Yes, with the Docker version in multi-user mode. The desktop app is single user only.
Can I turn off AnythingLLM telemetry?
Yes. Disable it in the sidebar under Privacy, or set DISABLE_TELEMETRY to true in the .env file.




