Module 11 — Evaluation and Quality

The difference between an amateur and an expert isn’t fancier indexes — it’s that the expert measures. “It seems to work” is how systems silently degrade. This module gives you the metrics and methods to know, with numbers, whether retrieval is good and whether a change helped.

Two different questions, two kinds of evaluation

Be careful not to confuse these:

1. System quality: is my ANN index returning the true nearest neighbors? (Measured by recall@k vs. exact search.) This is about the index doing its job correctly.
2. Retrieval quality: are the returned items actually relevant and useful to the user/LLM? (Measured by ranking metrics like MRR, nDCG, precision/recall against human relevance judgments.) This is about whether “nearest by embedding” means “good answer.”

You can have perfect recall@k (your index nails the true neighbors) and still terrible retrieval quality (because your embeddings or chunking are bad). Both matter; they’re fixed in different places.

System quality: recall@k (revisited, precisely)

recall@k = (true top-k neighbors your index returned) / k

You need ground truth = the exact top-k from brute force (Module 4) on a representative query sample. Then sweep your index dials (efSearch, nprobe) and plot recall vs. latency. Pick the cheapest setting that meets your recall target. Typical production targets: 0.95–0.99. This tells you if approximation/compression is costing too much accuracy.

Retrieval quality: the ranking metrics

These need relevance judgments — for a set of test queries, which documents are actually relevant (from human labels, click logs, or known answers). Then:

Precision@k and Recall@k (information-retrieval sense)

• Precision@k = of the k results you showed, what fraction are relevant? (“Don’t show junk.”)
• Recall@k = of all relevant docs that exist, what fraction did you retrieve in k? (“Don’t miss things.”)

(Note: “recall” is overloaded — IR recall vs. ANN recall. Context tells you which.)

MRR (Mean Reciprocal Rank) — “how high is the first good hit?”

For each query, find the rank of the first relevant result; score = 1/rank (1st → 1.0, 2nd → 0.5, 3rd → 0.33…). Average over queries. Great when there’s essentially one right answer (Q&A, “find the doc”). Rewards putting the right thing at the top.

nDCG (Normalized Discounted Cumulative Gain) — the gold standard for ranking

Handles graded relevance (some results are very relevant, some slightly) and rewards putting the most relevant items highest, with a logarithmic discount for lower positions. Normalized to 0–1 against the ideal ordering. Use when ranking order and degrees of relevance matter — i.e., most real search. Most rigorous; slightly more work to set up (needs graded labels).

Hit Rate / Recall@k for RAG

Simple and popular for RAG: for each test question with a known answer-containing chunk, did it appear in the top-k retrieved? Fraction of questions where it did = hit rate@k. Quick, intuitive, and a strong first metric for “is retrieval feeding the LLM the right context?”

A quick map of which to use

Situation	Metric
Is my ANN index accurate vs exact?	recall@k vs brute-force
One correct answer (Q&A, lookup)	MRR, hit rate@k
Graded relevance, ranking order matters	nDCG
”Don’t show junk” / “don’t miss things”	precision@k / recall@k
End-to-end RAG answer quality	answer faithfulness/correctness (below)

Building an evaluation set (the work that pays off most)

You cannot evaluate without test data. Options, roughly in order of effort and value:

1. Hand-label a few dozen to a few hundred real queries with their relevant documents. Tedious but gold. Even 50 good examples beats zero.
2. Mine click/usage logs — what users clicked or marked helpful = implicit relevance labels. Scales well once you have traffic.
3. LLM-generated eval sets — have an LLM read each chunk and generate questions it answers; the source chunk is the “correct” retrieval. Fast way to bootstrap hundreds of (question, gold-chunk) pairs. Validate a sample by hand — LLM-generated sets can be too easy or leak phrasing.
4. Public benchmarks (BEIR for retrieval, MTEB for embeddings) — useful for model selection, but always confirm on your own data; leaderboard rank ≠ performance on your domain.

The golden rule: evaluate on data that looks like your real production queries. A model that wins on benchmarks can lose on your jargon-heavy support tickets. Your eval set is your most valuable asset for improving the system.

Evaluating end-to-end RAG (beyond retrieval)

RAG has two stages to grade:

• Retrieval: did we fetch the right context? (hit rate, MRR, nDCG, context recall.)
• Generation: given that context, was the answer good? Key dimensions:
  • Faithfulness / groundedness: is every claim supported by the retrieved context (no hallucination)?
  • Answer relevance: does it actually address the question?
  • Context precision: was the retrieved context mostly relevant or padded with noise?
  • Correctness: does it match the known ground-truth answer?

LLM-as-judge is the common scalable method: a strong model scores answers on these dimensions against the context and/or a reference answer. Frameworks like Ragas, TruLens, DeepEval, ARES automate this. Calibrate the judge against some human labels — LLM judges have biases (length, position, self-preference).

The experiment loop (how experts actually improve systems)

1. Build/refresh an eval set from real queries.
2. Measure a baseline (retrieval metrics + RAG metrics).
3. Change ONE thing (chunk size, embedding model, add hybrid, add reranker, tune ef).
4. Re-measure on the SAME eval set.
5. Keep if it helps, revert if not. Log the result.
6. Repeat.

Change one variable at a time, hold the eval set fixed, and let numbers — not intuition — decide. This discipline is the whole job. Most “obvious” improvements turn out neutral or negative when measured; that’s exactly why you measure.

Hands-on: compute the core metrics

def recall_at_k(retrieved, relevant, k):
    hits = sum(1 for r in retrieved[:k] if r in relevant)
    return hits / max(1, min(k, len(relevant)))

def precision_at_k(retrieved, relevant, k):
    return sum(1 for r in retrieved[:k] if r in relevant) / k

def reciprocal_rank(retrieved, relevant):
    for i, r in enumerate(retrieved):
        if r in relevant:
            return 1.0 / (i + 1)
    return 0.0

def hit_rate(retrieved, relevant, k):
    return 1.0 if any(r in relevant for r in retrieved[:k]) else 0.0

# Average each over your whole eval set to get MRR, mean recall@k, hit rate@k, etc.
# nDCG: use sklearn.metrics.ndcg_score with graded relevance labels.

Run these over your eval set before and after every change. That number moving is the only proof that matters.

Gotchas

• No eval set. The cardinal sin. Without it you’re guessing, and confident guesses compound into bad systems.
• Confusing ANN recall with IR relevance. Perfect index recall + bad embeddings = bad results. Diagnose which layer is failing.
• Optimizing the wrong metric. MRR for a multi-answer task, or precision when missing items is the real problem. Match metric to user need.
• Evaluating on toy/benchmark data only. Real queries are messier; validate on production-like inputs.
• Trusting LLM-judge blindly. Calibrate against human labels; watch for length/position bias.
• Changing many things at once. You won’t know what helped. One variable per experiment.
• Eval set leakage / staleness. If your eval questions are too similar to your chunks (LLM-generated) it’s too easy; refresh and include hard, real cases.

Check yourself

1. What’s the difference between ANN recall@k and retrieval (IR) quality? Can one be perfect while the other is poor?
2. When would you choose MRR over nDCG?
3. Why is your own eval set more valuable than a public leaderboard?
4. Name the four dimensions for grading a RAG answer (not just retrieval).
5. What’s the one-variable-at-a-time rule and why does it matter?

Answers: (1) ANN recall measures whether the index returned the true nearest neighbors; IR quality measures whether returned items are actually relevant/useful. Yes — a perfect index over bad embeddings yields perfect recall@k but poor relevance. (2) When there’s essentially one correct answer and you care most about how high the first hit ranks (Q&A/lookup); nDCG is for graded relevance and overall ranking order. (3) It reflects your real domain, jargon, and query patterns; benchmark winners often underperform on your specific data. (4) Faithfulness/groundedness, answer relevance, context precision, and correctness. (5) Change one thing, re-measure on a fixed eval set, keep or revert based on numbers — so you actually know which change caused the effect.

➡️ Next: 12-advanced-and-frontier.md — reranking, multi-vector, GraphRAG, and the cutting edge.