Module 08 — Embeddings & Vector Databases

Now the part that feels like magic but isn’t: how a computer searches by meaning instead of by exact words. This is the engine of RAG retrieval. We’ll build the intuition carefully — no math background needed.

1. The problem with keyword search

Old-fashioned search matches words. Search “car” and you miss documents that say “automobile,” “vehicle,” or “sedan.” Search “how do I get my money back” and a document titled “Refund Policy” might not contain those exact words at all.

We want search that understands meaning: “car” and “automobile” should match because they mean the same thing. That’s what embeddings give us.

2. What is an embedding?

An embedding is a list of numbers (called a vector) that represents the meaning of a piece of text. A special model — an embedding model — reads a chunk of text and outputs, say, 768 or 1,536 numbers that capture “what this text is about.”

The key property:

Texts with similar meaning get similar number-lists. Texts with different meaning get different number-lists.

So “car” and “automobile” produce nearly identical vectors; “car” and “banana” produce very different ones — even though “car” and “banana” are both short common words.

The map analogy

Picture a giant map where every possible piece of text is a dot. The embedding model places each text on the map so that related ideas sit close together and unrelated ideas sit far apart. All the “refund/return/money-back” texts cluster in one neighborhood; all the “shipping/delivery” texts cluster in another.

This map isn’t 2-D like a paper map — it has hundreds of dimensions (that’s what all those numbers are). You can’t picture hundreds of dimensions, and you don’t need to. Just hold the intuition: embedding = a position on a meaning-map; similar meanings are neighbors.

3. How searching works, then

Here’s the whole trick of semantic search:

Ahead of time (indexing): embed every chunk → each chunk becomes a dot on the meaning-map. Store all the dots.
At question time: embed the question the same way → the question becomes a dot too.
Find the nearest dots. The chunks closest to the question on the map are the most meaning-relevant. Return those.

“Closeness” is measured with a similarity score (the common one is cosine similarity — you don’t need the formula; just know it’s a number where higher = more similar in meaning).

That’s it. Semantic search = embed everything, embed the question, return the nearest neighbors. This is why a question phrased totally differently from the document can still find it — they land near each other on the meaning-map.

4. Choosing an embedding model (it’s a real decision)

Not all embedding models are equal, and the default is rarely optimal. Things that matter:

Domain. A general model may be mediocre on legal, medical, or code text. Specialized models exist.
Language. A model trained mostly on English will retrieve poorly on other languages. For multilingual content, use a multilingual embedding model — otherwise a French question won’t find French documents well.
Max input length. Each embedding model can only read up to so many tokens at once. Your chunks must fit — if a chunk is longer than the model’s limit, the extra is silently cut off and that meaning is lost. (This is why chunk size and embedding model are linked decisions.)
Vector size (dimensions). More numbers can mean richer meaning but more storage and slightly slower search. A trade-off.
Query vs document encoding. Some models expect you to label inputs (e.g. prefix questions with “query:” and passages with “passage:”). Getting this wrong quietly halves quality. Check the model’s instructions.

Practical tip: leaderboards (like MTEB) are a starting point, but the only test that counts is how a model performs on your data and your questions (Module 12). Pick two or three candidates and measure.

One more rule that bites people: the embedding model is a dependency you’ve committed to. If you later switch models, all your stored vectors are in a different “language” and become meaningless next to new ones — you must re-embed your entire library. Plan for that.

5. Where the vectors live — the vector database

You might have millions of chunk-vectors. At question time you need the nearest ones fast. Checking the question against every single vector one by one would be too slow at scale. So we use a vector database (or vector store) with a clever index.

Approximate search (ANN)

These databases use Approximate Nearest Neighbor (ANN) search. “Approximate” is the key word:

To be fast, the database doesn’t guarantee it finds the absolute closest matches — it finds almost certainly the closest ones, trading a tiny bit of accuracy for enormous speed.

You don’t need the algorithms, but you’ll hear these names:

HNSW — a popular, fast, accurate method (uses more memory). The common default.
IVF / quantization — methods that save memory by grouping or compressing vectors, with a small accuracy cost.

The practical knob is recall vs speed: you can tune the index to be more thorough (slower, finds more of the true nearest) or faster (might miss a few). Recall here means “what fraction of the truly-relevant chunks did we actually find.”

Which vector store?

Options range from dedicated databases (Pinecone, Weaviate, Qdrant, Milvus) to adding vectors to a database you already run (e.g. pgvector for PostgreSQL) to search engines (Elasticsearch/OpenSearch) to simple libraries (FAISS) for small projects. How to choose (Module 15 expands):

Scale — thousands vs billions of chunks.
Filtering — do you need to combine vector search with metadata filters (date, permissions)? Most do.
Hybrid search — does it also do keyword search? (Module 09 — you’ll want this.)
Operational burden — a new database to run, or reuse what you have.

Sensible default for many teams: if you already run PostgreSQL, pgvector is often enough. Don’t adopt an exotic database until your scale demands it.

6. The limitation that sets up the next module

Semantic/embedding search is wonderful at meaning… and surprisingly bad at exact matches. Because it works on fuzzy meaning, it can fumble:

Product codes, SKUs, error codes (“ERR_4042”).
Specific names, acronyms, rare technical terms.

The embedding “smears” these into nearby concepts and may miss the exact one. The fix is to combine meaning-search with old-fashioned keyword-search — hybrid search — which is exactly where Module 09 begins.

Check yourself

In one sentence, what is an embedding?
Explain semantic search using the meaning-map idea.
Why must your chunk size fit your embedding model’s input limit?
What does “approximate” mean in Approximate Nearest Neighbor, and why accept approximation?
Name one kind of query that pure embedding search tends to handle badly.

Next: Module 09 — Retrieval, Hybrid Search & Reranking →