Embeddings and vector search explained

Keyword search finds exact words. If a user searches "how to fix a slow site" and your article says "improve page speed", keyword search may miss it. Embeddings let you search by meaning.

What is an embedding?

An embedding is a list of numbers (a vector) that represents a piece of text, produced by an embedding model. Typical sizes are a few hundred to a few thousand numbers.

The key property: texts with similar meaning get vectors that are close together. "Car" and "automobile" end up near each other; "car" and "banana" far apart. Imagine every sentence as a point on a huge map where neighbours mean similar things.

Embeddings also exist for images, audio and code, and some models place text and images in the same space.

Measuring similarity

To compare two vectors we measure distance or angle:

Cosine similarity: the angle between vectors. 1 means same direction (very similar), 0 unrelated. The most common choice for text.
Dot product: like cosine when vectors are normalised.
Euclidean distance: straight-line distance.

Use the metric your embedding model recommends, and the same one for indexing and querying.

The search flow

Index time: split your documents into pieces, embed each piece and store the vector with the text and metadata.
Query time: embed the user's question with the same model, find the nearest stored vectors, return the matching pieces.

-- PostgreSQL with the pgvector extension
CREATE EXTENSION vector;
CREATE TABLE chunks (
  id bigserial PRIMARY KEY,
  doc_id int,
  content text,
  embedding vector(768)
);

SELECT id, content
FROM chunks
ORDER BY embedding <=> '[0.12, -0.03, ...]'   -- cosine distance
LIMIT 5;

(The vector size depends on your embedding model.)

Chunking

Documents are usually too long to embed whole, and one vector for a long text blurs its meaning. Split into chunks:

Roughly a few hundred tokens each, often by paragraph or section so ideas stay together.
A small overlap between chunks avoids cutting an idea in half.
Store the title and source with each chunk, so results can be explained and linked.

Chunk size is a trade-off: too small loses context, too large dilutes meaning. Test on your own data.

Approximate nearest neighbour indexes

Comparing a query to millions of vectors one by one is slow. Vector indexes such as HNSW and IVF find approximately the nearest vectors quickly, trading a little accuracy for large speed gains. Tune them and measure recall.

Where to store vectors

pgvector in PostgreSQL: simplest if you already use Postgres, and lets you filter with normal SQL.
Dedicated vector databases: Pinecone, Qdrant, Weaviate, Milvus, Chroma and others, useful at large scale or for special features.
Some search engines (Elasticsearch, OpenSearch) also support vectors.

For a small or medium project, start with what you already run.

Hybrid search

Meaning-based search can miss exact terms like product codes, names or error numbers. Hybrid search combines keyword search (such as BM25) with vector search and merges the rankings. It often beats either one alone. A re-ranker model can then reorder the top results for better accuracy.

Metadata filters

Combine vector search with filters: language, date, category, or which user is allowed to see the document. Apply access filters in the query itself, so people cannot retrieve content they have no right to see.

Common mistakes

Mixing embeddings from different models in one index.
Changing the embedding model without re-embedding everything.
Chunks that are too large, or that cut sentences in the middle.
No evaluation: build a small set of questions with known right answers and check whether they come back.
Ignoring permissions on retrieved content.
Sending private text to an embedding service without checking its data policy.

Where it is used

Semantic site search, recommendations ("similar articles"), duplicate detection, clustering, and retrieval-augmented generation (RAG), covered in the next article.

Takeaway: embeddings map meaning to vectors, nearest-neighbour search finds related text, good chunking and metadata make it useful, and hybrid search plus evaluation make it reliable.

AI & LLM series, Part 1 (building with language models): 3 of 5. Previous: Prompt engineering: a practical guide to better LLM results.

Embeddings and vector search explained for developers