Ollama isn’t just for text. With a vision model, your local AI can look at a photo, read text in a screenshot, describe a chart, check a receipt or explain an error message, all without sending the image to the cloud.
It takes one command to try. Point the model at an image file in the terminal, or send the image through the API from Python, JavaScript or any app that talks to Ollama.
This guide shows how to run a vision model in Ollama, pass images in the CLI, the REST API, Python and JavaScript, use them in a chat UI, pick the right model for your GPU, and fix the common problems. Commands follow the official Ollama docs as of November 2026.
- Try it:
ollama run gemma4 ./image.png what is in this image? - CLI: put the image path in the prompt; jpg and png work.
- REST API: POST to
http://localhost:11434/api/chatwith an"images"array of base64 strings. - Python/JS libraries: pass a file path, raw bytes or base64 in
images. - Chat UI: Open WebUI lets you drop images into the chat.
- Raise context if long answers about big images get cut off.
What a Vision Model Can Do
A vision (multimodal) model accepts images as well as text. You give it a picture plus a question, and it answers about what it sees.
| Task | Example |
|---|---|
| Describe images | “Describe this photo for alt text” |
| Read text (OCR) | “What does the sign say?” |
| Explain screenshots | “What does this error mean and how do I fix it?” |
| Charts and tables | “Summarise the trend in this chart” |
| Documents | “List the items and total on this receipt” |
| Sorting photos | “Is this a cat, a dog or neither? Answer with one word” |
Everything runs on your own PC, which is great for private photos and documents. New to Ollama? Start with our complete Ollama guide.
Which Vision Model to Use
The current Ollama docs use gemma4 as the example vision model. Ollama’s library has a Vision filter that lists every model that accepts images; check it for the latest options and sizes.
| Model | Command | Notes |
|---|---|---|
| Gemma 4 | ollama run gemma4 |
Used in the official vision docs |
| LLaVA 1.6 | ollama run llava:7b / llava:13b / llava:34b |
Older classic; Ollama’s 2024 post notes up to 4x more pixels than v1.5 and better text reading |
| Other vision models | See the Vision filter on ollama.com | Pick a size that fits your VRAM |
VRAM rule of thumb: pick the smallest model that does the job well. Images add extra tokens to the context, so leave headroom. If you run out, see our Ollama not using GPU fix.
Use Images in the Ollama CLI
- Open a terminal.
- Run the model with an image path in the prompt:
ollama run gemma4 ./image.png what is in this image? - Wait for the download the first time;
ollama runpulls the model automatically. - Read the answer, then ask follow-up questions in the same session.
You can also put the path inside a quoted prompt, as in Ollama’s original example:
ollama run llava "describe this image: ./art.jpg"
Use .jpg or .png files. On Windows, a full path such as C:/Users/you/Pictures/photo.jpg works; forward slashes avoid escaping problems.
Send Images Through the REST API
The REST API needs the image as a base64 string inside an images array on the message. This is the pattern from the official docs:
curl -L -o test.jpg "https://upload.wikimedia.org/wikipedia/commons/3/3a/Cat03.jpg"
IMG=$(base64 < test.jpg | tr -d '\n')
curl -X POST http://localhost:11434/api/chat \
-H "Content-Type: application/json" \
-d '{
"model": "gemma4",
"messages": [{
"role": "user",
"content": "What is in this image?",
"images": ["'"$IMG"'"]
}],
"stream": false
}'
"stream": falsereturns one JSON response instead of a stream.- The answer is in
message.content. - The SDKs (below) accept file paths, URLs or bytes; only the raw REST API needs base64.
More API basics are in our Ollama API guide.
Python Example
Install the library with pip install ollama, then:
from ollama import chat
path = input('Path to an image: ')
response = chat(
model='gemma4',
messages=[{
'role': 'user',
'content': 'What is in this image? Be concise.',
'images': [path],
}],
)
print(response.message.content)
Instead of a path you can pass raw bytes (Path(path).read_bytes()) or a base64 string (base64.b64encode(Path(path).read_bytes()).decode()), which is handy when the image comes from a web upload.
Batch-describe a folder of photos
from pathlib import Path
from ollama import chat
for img in Path('photos').glob('*.jpg'):
r = chat(model='gemma4', messages=[{
'role': 'user',
'content': 'Write one short alt-text sentence for this image.',
'images': [str(img)],
}])
print(img.name, '-', r.message.content)
JavaScript Example
Install with npm install ollama, then:
import ollama from 'ollama'
const imagePath = '/absolute/path/to/image.jpg'
const response = await ollama.chat({
model: 'gemma4',
messages: [
{ role: 'user', content: 'What is in this image?', images: [imagePath] }
],
stream: false,
})
console.log(response.message.content)
Use Vision in a Chat Interface
Prefer a ChatGPT-style window? Open WebUI connects to Ollama and lets you attach images to a message. Pick your vision model, click the attach button or drag the picture in, and ask your question. Our Open WebUI with Ollama setup guide walks through installation.
Want a model that always answers in a set style, such as “describe images for blind users”? Bake the instructions in with a Modelfile.
Practical Tasks and Prompts
| Task | Prompt |
|---|---|
| Alt text | “Write a 15-word alt text for this image. No ‘image of’.” |
| Receipt | “List each item with its price as a table, then the total.” |
| Screenshot error | “Read the error in this screenshot and suggest a fix.” |
| Chart | “What are the three main takeaways from this chart?” |
| Handwriting | “Transcribe this handwritten note.” |
| Product photo | “Write a 50-word product description from this photo.” |
| Classification | “Answer only ‘indoor’ or ‘outdoor’.” |
Working with PDFs rather than pictures? See how to chat with PDFs locally.
Tips for Better Results
- Be specific. “Read the total on this receipt” beats “what is this?”
- Crop to what matters. Small text in a huge photo is harder to read.
- Use clear, well-lit images. Blurry photos mean guessy answers.
- Ask for a format: table, JSON, one word, bullet list.
- Double-check numbers. Vision models can misread digits; verify anything important.
- Increase context length for long outputs. See our Ollama context length guide.
Local Vision Model vs OCR Tool vs Cloud AI
| Ollama vision model | Classic OCR tool | Cloud AI (ChatGPT, Gemini) | |
|---|---|---|---|
| Privacy | Images stay on your PC | Usually local | Images are uploaded |
| Understands meaning | Yes, can explain and reason | No, only extracts text | Yes, usually strongest |
| Cost | Free after download | Free or cheap | Free tiers with limits, or paid |
| Speed | Depends on your GPU | Very fast | Fast, needs internet |
| Best for | Private photos, documents, batch jobs | Pure text extraction from clean scans | Hard images where accuracy matters most |
Our take: use a local vision model for anything private or for large batches you’d rather not pay per image for. Use a dedicated OCR tool when you only need clean text from scans. Use a cloud model as a second opinion on difficult images.
Troubleshooting
| Problem | Likely cause | Fix |
|---|---|---|
| Model ignores the image | Text-only model | Use a model with vision support (check the Vision filter) |
| “File not found” or no image detected | Wrong path | Use a full path; forward slashes on Windows |
| API returns an error about images | Not base64, or line breaks in the string | Base64-encode and strip newlines (tr -d '\n') |
| Very slow | Model on CPU or too big | Use a smaller model; check GPU use with ollama ps |
| Out of memory | Model plus image too large for VRAM | Smaller model or quantization |
| Answer cut off | Context too small | Raise num_ctx |
| Wrong text read | Low resolution or blur | Crop and use a sharper image |
| WebP or HEIC not working | Format not supported | Convert to jpg or png |
FAQ
Can Ollama read images?
Yes, with a vision model such as gemma4 or LLaVA. Pass the image path in the CLI or send it through the API.
What image formats does Ollama support?
Ollama’s docs mention .jpg and .png. Convert other formats first.
Does it work offline?
Yes. After the model downloads, everything runs locally and your images never leave your PC.
How do I send an image with the API?
Add an images array with a base64 string to your message when you POST to /api/chat.
Can I send several images at once?
The images field is an array, but Ollama’s docs only show one image. Support for several depends on the model, so test with yours.
Which vision model is best?
It depends on your GPU and task. Start with the model used in Ollama’s docs, then compare others from the Vision filter on your own images.




