Skip to content

Ollama Embeddings (2026): Generate Vectors Locally With CLI, API and Python

6–9 minutes
Laughing woman in a black turban and white jacket by a night city window holding a card with the Ollama llama logo, with the text Local AI Search, No API Key

Embeddings turn text into lists of numbers that capture meaning. Two sentences that mean the same thing end up with similar numbers, even if they use different words. That simple idea powers semantic search, “chat with your documents” (RAG), recommendations, duplicate detection and clustering.

Ollama can create embeddings locally, for free, with no API key and no data leaving your machine. You pull a small embedding model, send it text, and get back vectors you can store and compare.

This guide shows how to generate embeddings with the Ollama CLI, the REST API, Python and JavaScript, how to batch inputs and shrink dimensions, how to compare vectors with cosine similarity, how to build a tiny semantic search, how to use the OpenAI-compatible endpoint, and how to fix common problems. Everything follows the official Ollama docs as of November 2026.

TL;DR

  • Model: Ollama’s docs use embeddinggemma. Get it with ollama pull embeddinggemma.
  • Quick test: ollama run embeddinggemma "Hello world" prints a JSON array.
  • API: POST http://localhost:11434/api/embed with model and input (a string or an array of strings).
  • Vectors are L2-normalized, so cosine similarity is just a dot product.
  • Always use the same model for indexing and for queries.
  • OpenAI apps: use http://localhost:11434/v1/embeddings with any API key string.

What Are Embeddings?

An embedding is a vector, a long list of numbers. Ollama’s docs say the vector length depends on the model, typically 384 to 1024 dimensions. Texts with similar meaning point in similar directions, so you can measure how related they are.

Use case Example
Semantic search Search notes for “how do I reset my router” and find “restarting the Wi-Fi box”
RAG Find the right document chunks to give an LLM before it answers
Duplicate detection Spot support tickets that ask the same thing
Clustering Group feedback into topics automatically
Recommendations “Articles similar to this one”

Embedding models don’t chat. They only turn text into vectors, so they’re small and fast.

Set Up an Embedding Model

  1. Install Ollama if you haven’t. Our Ollama guide covers Windows, Mac and Linux.
  2. Pull the model: ollama pull embeddinggemma
  3. Check it’s there: ollama list
  4. Keep Ollama running in the background (the app or ollama serve).

Ollama’s library has a filter for embedding models if you want to compare others. Whatever you choose, stick with it: vectors from different models can’t be compared.

Embeddings From the CLI

ollama run embeddinggemma "Hello world"

You can also pipe text in:

echo "Hello world" | ollama run embeddinggemma

The output is a JSON array of numbers. The CLI is handy for testing; real apps use the API.

The /api/embed Endpoint

curl http://localhost:11434/api/embed -d '{
  "model": "embeddinggemma",
  "input": "Why is the sky blue?"
}'

Request fields

Field Required What it does
model Yes Embedding model name
input Yes A string, or an array of strings for batches
truncate No (default true) Cut inputs longer than the context window; if false, return an error instead
dimensions No Number of dimensions to generate
keep_alive No How long to keep the model loaded
options No Runtime options such as num_ctx

Response fields

Field Meaning
model The model used
embeddings An array of vectors, one per input
total_duration, load_duration Timings in nanoseconds
prompt_eval_count Number of input tokens processed

Batch and dimension examples

curl http://localhost:11434/api/embed -d '{
  "model": "embeddinggemma",
  "input": ["Why is the sky blue?", "Why is the grass green?"]
}'

curl http://localhost:11434/api/embed -d '{
  "model": "embeddinggemma",
  "input": "Generate embeddings for this text",
  "dimensions": 128
}'

Smaller dimensions mean smaller storage and faster search, at some cost in accuracy. Only use it with models that support reduced dimensions. More on the API: Ollama API guide.

Python Examples

Install the library: pip install ollama

import ollama

single = ollama.embed(
    model='embeddinggemma',
    input='The quick brown fox jumps over the lazy dog.'
)
print(len(single['embeddings'][0]))

batch = ollama.embed(
    model='embeddinggemma',
    input=[
        'The quick brown fox jumps over the lazy dog.',
        'The five boxing wizards jump quickly.',
        'Jackdaws love my big sphinx of quartz.',
    ]
)
print(len(batch['embeddings']))

The first print shows the vector length; the second shows one vector per input.

JavaScript Example

Install: npm install ollama

import ollama from 'ollama'

const single = await ollama.embed({
  model: 'embeddinggemma',
  input: 'The quick brown fox jumps over the lazy dog.',
})
console.log(single.embeddings[0].length)

const batch = await ollama.embed({
  model: 'embeddinggemma',
  input: ['First sentence.', 'Second sentence.', 'Third sentence.'],
})
console.log(batch.embeddings.length)

Because Ollama returns unit-length vectors, cosine similarity is the dot product. This script finds the note closest in meaning to a question:

import ollama

notes = [
    'Restart the Wi-Fi router by unplugging it for 30 seconds.',
    'Our office is closed on public holidays.',
    'To change your password, open Settings and choose Security.',
    'Invoices are emailed on the first working day of each month.',
]

def embed(texts):
    return ollama.embed(model='embeddinggemma', input=texts)['embeddings']

note_vectors = embed(notes)

def search(question, top_k=2):
    q = embed([question])[0]
    scores = [sum(a * b for a, b in zip(q, v)) for v in note_vectors]
    ranked = sorted(zip(scores, notes), reverse=True)
    return ranked[:top_k]

for score, note in search('my internet is not working'):
    print(round(score, 3), note)
  1. Embed your documents once and save the vectors.
  2. Embed each question with the same model.
  3. Score every document with the dot product.
  4. Return the top matches.

For thousands of documents, store vectors in a vector database instead of a Python list.

OpenAI-Compatible Endpoint

Apps built for OpenAI’s embeddings API can point at Ollama:

from openai import OpenAI

client = OpenAI(base_url='http://localhost:11434/v1/', api_key='ollama')
resp = client.embeddings.create(model='embeddinggemma', input=['Hello world'])
print(len(resp.data[0].embedding))
  • Supported fields: model, input (string or array of strings), encoding_format and dimensions.
  • Token arrays and the user field aren’t supported.
  • The API key is required by the SDK but ignored by Ollama, so any string works.

Storing Vectors: Size and Database Options

Each number in a vector is usually stored as a 4-byte float. A 768-dimension vector is about 3 KB, so 100,000 chunks need roughly 300 MB before indexes. Using the dimensions field to go smaller cuts this proportionally.

Storage Good for
Python list or NumPy file Experiments and a few thousand texts
SQLite with a vector extension Simple apps in a single file
Chroma Local prototypes and RAG apps
Qdrant Fast local or server search with filters
PostgreSQL with pgvector Apps that already use Postgres

Whatever you pick, save the original text, the source (file name or URL) and the embedding model name next to each vector, so you can show citations and re-embed later.

Using Embeddings for RAG

RAG (retrieval augmented generation) uses embeddings to find relevant chunks, then a chat model answers using them.

  1. Split documents into chunks of a few hundred words.
  2. Embed every chunk and store vectors with the chunk text.
  3. Embed the user’s question and fetch the closest chunks.
  4. Send the chunks plus the question to a chat model in Ollama.

Don’t want to code it? Open WebUI RAG does all of this in a UI, and our LangChain with Ollama guide shows a code version. For PDFs, see chat with PDFs locally.

Best Practices

  • One model per index. Re-embed everything if you switch models.
  • Batch inputs by passing an array; it’s much faster than one request per text.
  • Use cosine similarity for semantic search, as Ollama’s docs recommend.
  • Chunk long documents. With truncate on, text past the context window is silently cut.
  • Keep the model loaded during big jobs with keep_alive. See Ollama keep alive.
  • Store the model name alongside your vectors so you know how they were made.

Troubleshooting

Problem Likely cause Fix
“model not found” Model not pulled Run ollama pull embeddinggemma
Connection refused Ollama not running Start the app or ollama serve
Error about input length truncate set to false Chunk the text or allow truncation
Search results look random Mixed models or dimensions Re-embed everything with one model and one dimension size
Chat model gives bad embeddings Using a chat model for embeddings Use a dedicated embedding model
Slow on big batches Model reloading Batch inputs and raise keep_alive
OpenAI SDK error about API key Key missing Pass any string, e.g. api_key='ollama'
Running on CPU GPU not detected See Ollama not using GPU

FAQ

Which Ollama model is best for embeddings?

Ollama’s docs use embeddinggemma. Check the embedding filter in Ollama’s library for others, and test on your own data.

What’s the difference between /api/embed and /api/embeddings?

/api/embed is the current endpoint documented by Ollama, with batch input and normalized vectors. Use it for new projects.

Are Ollama embeddings normalized?

Yes. The /api/embed endpoint returns L2-normalized (unit-length) vectors.

Can I embed many texts in one request?

Yes. Pass an array of strings to input and you get one vector per string.

Do Ollama embeddings work offline?

Yes. After you pull the model, everything runs locally.

Can I use Ollama embeddings with OpenAI libraries?

Yes. Set the base URL to http://localhost:11434/v1/ and use any API key string.

Sources

Free weekly email

Get new AI guides every week

One short weekly email with our newest local AI, ComfyUI and AI tool guides. No spam, no pop-ups, just the good stuff.

Free. Unsubscribe anytime in one click. We only use your email to send this newsletter. See our privacy policy.

Laughing woman in a black turban and white jacket by a night city window holding a card with the Ollama llama logo, with the text Local AI Search, No API Key