Embeddings turn text into lists of numbers that capture meaning. Two sentences that mean the same thing end up with similar numbers, even if they use different words. That simple idea powers semantic search, “chat with your documents” (RAG), recommendations, duplicate detection and clustering.
Ollama can create embeddings locally, for free, with no API key and no data leaving your machine. You pull a small embedding model, send it text, and get back vectors you can store and compare.
This guide shows how to generate embeddings with the Ollama CLI, the REST API, Python and JavaScript, how to batch inputs and shrink dimensions, how to compare vectors with cosine similarity, how to build a tiny semantic search, how to use the OpenAI-compatible endpoint, and how to fix common problems. Everything follows the official Ollama docs as of November 2026.
- Model: Ollama’s docs use
embeddinggemma. Get it withollama pull embeddinggemma. - Quick test:
ollama run embeddinggemma "Hello world"prints a JSON array. - API: POST
http://localhost:11434/api/embedwithmodelandinput(a string or an array of strings). - Vectors are L2-normalized, so cosine similarity is just a dot product.
- Always use the same model for indexing and for queries.
- OpenAI apps: use
http://localhost:11434/v1/embeddingswith any API key string.
What Are Embeddings?
An embedding is a vector, a long list of numbers. Ollama’s docs say the vector length depends on the model, typically 384 to 1024 dimensions. Texts with similar meaning point in similar directions, so you can measure how related they are.
| Use case | Example |
|---|---|
| Semantic search | Search notes for “how do I reset my router” and find “restarting the Wi-Fi box” |
| RAG | Find the right document chunks to give an LLM before it answers |
| Duplicate detection | Spot support tickets that ask the same thing |
| Clustering | Group feedback into topics automatically |
| Recommendations | “Articles similar to this one” |
Embedding models don’t chat. They only turn text into vectors, so they’re small and fast.
Set Up an Embedding Model
- Install Ollama if you haven’t. Our Ollama guide covers Windows, Mac and Linux.
- Pull the model:
ollama pull embeddinggemma - Check it’s there:
ollama list - Keep Ollama running in the background (the app or
ollama serve).
Ollama’s library has a filter for embedding models if you want to compare others. Whatever you choose, stick with it: vectors from different models can’t be compared.
Embeddings From the CLI
ollama run embeddinggemma "Hello world"
You can also pipe text in:
echo "Hello world" | ollama run embeddinggemma
The output is a JSON array of numbers. The CLI is handy for testing; real apps use the API.
The /api/embed Endpoint
curl http://localhost:11434/api/embed -d '{
"model": "embeddinggemma",
"input": "Why is the sky blue?"
}'
Request fields
| Field | Required | What it does |
|---|---|---|
model |
Yes | Embedding model name |
input |
Yes | A string, or an array of strings for batches |
truncate |
No (default true) | Cut inputs longer than the context window; if false, return an error instead |
dimensions |
No | Number of dimensions to generate |
keep_alive |
No | How long to keep the model loaded |
options |
No | Runtime options such as num_ctx |
Response fields
| Field | Meaning |
|---|---|
model |
The model used |
embeddings |
An array of vectors, one per input |
total_duration, load_duration |
Timings in nanoseconds |
prompt_eval_count |
Number of input tokens processed |
Batch and dimension examples
curl http://localhost:11434/api/embed -d '{
"model": "embeddinggemma",
"input": ["Why is the sky blue?", "Why is the grass green?"]
}'
curl http://localhost:11434/api/embed -d '{
"model": "embeddinggemma",
"input": "Generate embeddings for this text",
"dimensions": 128
}'
Smaller dimensions mean smaller storage and faster search, at some cost in accuracy. Only use it with models that support reduced dimensions. More on the API: Ollama API guide.
Python Examples
Install the library: pip install ollama
import ollama
single = ollama.embed(
model='embeddinggemma',
input='The quick brown fox jumps over the lazy dog.'
)
print(len(single['embeddings'][0]))
batch = ollama.embed(
model='embeddinggemma',
input=[
'The quick brown fox jumps over the lazy dog.',
'The five boxing wizards jump quickly.',
'Jackdaws love my big sphinx of quartz.',
]
)
print(len(batch['embeddings']))
The first print shows the vector length; the second shows one vector per input.
JavaScript Example
Install: npm install ollama
import ollama from 'ollama'
const single = await ollama.embed({
model: 'embeddinggemma',
input: 'The quick brown fox jumps over the lazy dog.',
})
console.log(single.embeddings[0].length)
const batch = await ollama.embed({
model: 'embeddinggemma',
input: ['First sentence.', 'Second sentence.', 'Third sentence.'],
})
console.log(batch.embeddings.length)
Build a Tiny Semantic Search
Because Ollama returns unit-length vectors, cosine similarity is the dot product. This script finds the note closest in meaning to a question:
import ollama
notes = [
'Restart the Wi-Fi router by unplugging it for 30 seconds.',
'Our office is closed on public holidays.',
'To change your password, open Settings and choose Security.',
'Invoices are emailed on the first working day of each month.',
]
def embed(texts):
return ollama.embed(model='embeddinggemma', input=texts)['embeddings']
note_vectors = embed(notes)
def search(question, top_k=2):
q = embed([question])[0]
scores = [sum(a * b for a, b in zip(q, v)) for v in note_vectors]
ranked = sorted(zip(scores, notes), reverse=True)
return ranked[:top_k]
for score, note in search('my internet is not working'):
print(round(score, 3), note)
- Embed your documents once and save the vectors.
- Embed each question with the same model.
- Score every document with the dot product.
- Return the top matches.
For thousands of documents, store vectors in a vector database instead of a Python list.
OpenAI-Compatible Endpoint
Apps built for OpenAI’s embeddings API can point at Ollama:
from openai import OpenAI
client = OpenAI(base_url='http://localhost:11434/v1/', api_key='ollama')
resp = client.embeddings.create(model='embeddinggemma', input=['Hello world'])
print(len(resp.data[0].embedding))
- Supported fields:
model,input(string or array of strings),encoding_formatanddimensions. - Token arrays and the
userfield aren’t supported. - The API key is required by the SDK but ignored by Ollama, so any string works.
Storing Vectors: Size and Database Options
Each number in a vector is usually stored as a 4-byte float. A 768-dimension vector is about 3 KB, so 100,000 chunks need roughly 300 MB before indexes. Using the dimensions field to go smaller cuts this proportionally.
| Storage | Good for |
|---|---|
| Python list or NumPy file | Experiments and a few thousand texts |
| SQLite with a vector extension | Simple apps in a single file |
| Chroma | Local prototypes and RAG apps |
| Qdrant | Fast local or server search with filters |
| PostgreSQL with pgvector | Apps that already use Postgres |
Whatever you pick, save the original text, the source (file name or URL) and the embedding model name next to each vector, so you can show citations and re-embed later.
Using Embeddings for RAG
RAG (retrieval augmented generation) uses embeddings to find relevant chunks, then a chat model answers using them.
- Split documents into chunks of a few hundred words.
- Embed every chunk and store vectors with the chunk text.
- Embed the user’s question and fetch the closest chunks.
- Send the chunks plus the question to a chat model in Ollama.
Don’t want to code it? Open WebUI RAG does all of this in a UI, and our LangChain with Ollama guide shows a code version. For PDFs, see chat with PDFs locally.
Best Practices
- One model per index. Re-embed everything if you switch models.
- Batch inputs by passing an array; it’s much faster than one request per text.
- Use cosine similarity for semantic search, as Ollama’s docs recommend.
- Chunk long documents. With
truncateon, text past the context window is silently cut. - Keep the model loaded during big jobs with
keep_alive. See Ollama keep alive. - Store the model name alongside your vectors so you know how they were made.
Troubleshooting
| Problem | Likely cause | Fix |
|---|---|---|
| “model not found” | Model not pulled | Run ollama pull embeddinggemma |
| Connection refused | Ollama not running | Start the app or ollama serve |
| Error about input length | truncate set to false |
Chunk the text or allow truncation |
| Search results look random | Mixed models or dimensions | Re-embed everything with one model and one dimension size |
| Chat model gives bad embeddings | Using a chat model for embeddings | Use a dedicated embedding model |
| Slow on big batches | Model reloading | Batch inputs and raise keep_alive |
| OpenAI SDK error about API key | Key missing | Pass any string, e.g. api_key='ollama' |
| Running on CPU | GPU not detected | See Ollama not using GPU |
FAQ
Which Ollama model is best for embeddings?
Ollama’s docs use embeddinggemma. Check the embedding filter in Ollama’s library for others, and test on your own data.
What’s the difference between /api/embed and /api/embeddings?
/api/embed is the current endpoint documented by Ollama, with batch input and normalized vectors. Use it for new projects.
Are Ollama embeddings normalized?
Yes. The /api/embed endpoint returns L2-normalized (unit-length) vectors.
Can I embed many texts in one request?
Yes. Pass an array of strings to input and you get one vector per string.
Do Ollama embeddings work offline?
Yes. After you pull the model, everything runs locally.
Can I use Ollama embeddings with OpenAI libraries?
Yes. Set the base URL to http://localhost:11434/v1/ and use any API key string.



