Anti-Patterns: What NOT To Do
Module 13 — Anti-Patterns: What NOT To Do
An anti-pattern is a tempting-but-wrong approach — a common mistake that feels reasonable and bites you later. This module collects the expensive ones. Knowing these by heart is a big part of what makes someone a trusted RAG advisor: you stop teams from walking into the same traps over and over.
Each is written as mistake → why it’s wrong → what to do instead.
1. Fine-tuning to inject facts
Mistake: “Let’s fine-tune the model on our documents so it learns them.” Why wrong: facts get diluted, can’t be updated without retraining, can’t be cited, and hallucination may worsen (Module 04). Instead: use RAG for facts; fine-tune only for behavior/format.
2. Skipping evaluation
Mistake: shipping because the demo looked good. Why wrong: real traffic is messier than your demo; you can’t tell if changes help; it breaks silently in production. Instead: build a golden dataset and measure retrieval and generation separately (Module 12) before optimizing.
3. Stuffing a huge K into the context
Mistake: “Retrieve 50 chunks and send them all to the model.” Why wrong: distractors bury the relevant chunk (“lost in the middle”), lower answer quality, and cost more tokens. Instead: retrieve broad, rerank, feed only the best few (Module 09).
4. Naive fixed-size chunking with no overlap or structure
Mistake: blindly cutting every N characters. Why wrong: slices facts in half and ignores document structure. Instead: use overlap and structure-aware chunking; tune size on your eval set (Module 07).
5. Pure vector search, no keyword/hybrid
Mistake: relying only on embedding search. Why wrong: misses exact terms — SKUs, error codes, names, acronyms. Instead: hybrid search (dense + BM25) (Module 09).
6. No reranker
Mistake: sending raw retrieval results straight to the model. Why wrong: you leave the single biggest precision win on the table; the best chunk may be ranked low. Instead: add a cross-encoder reranker (Module 09).
7. Ignoring parsing quality
Mistake: dumping mangled PDF/table text into the pipeline. Why wrong: corrupted input poisons everything downstream — no prompt can fix scrambled data. Instead: use layout-aware parsing, handle tables, OCR scans, verify quality (Module 07).
8. Chunks bigger than the embedding model’s limit
Mistake: large chunks fed to an embedder with a smaller input limit. Why wrong: the excess is silently truncated, so part of the chunk’s meaning is lost from the index. Instead: match chunk size to the embedder’s max input length (Module 08).
9. No metadata / no filtering
Mistake: storing chunks with no tags. Why wrong: can’t filter by date/type, can’t cite precisely, and — critically — can’t enforce access control, so users may retrieve documents they shouldn’t see. Instead: attach metadata to every chunk and filter on it (Modules 07, 09, 14).
10. Stale index
Mistake: indexing once and never refreshing. Why wrong: RAG’s entire selling point is freshness — a rotting index quietly serves outdated answers. Instead: build an incremental indexing pipeline that adds/updates/deletes as documents change (Module 14).
11. No “I don’t know” path
Mistake: always forcing an answer. Why wrong: with no relevant context, the model confidently hallucinates. Instead: instruct it to abstain when the context lacks the answer (Module 10).
12. Using RAG for structured queries
Mistake: answering “total sales by region last quarter” via semantic search over prose. Why wrong: that’s a database/aggregation question, not a meaning-search question. Instead: use text-to-SQL or an API/tool call (Module 05).
13. Using RAG when the corpus fits in context
Mistake: building a retrieval stack for a small, static set of documents. Why wrong: over-engineering — you’ve added failure modes for no benefit. Instead: just paste it into the prompt (+ caching) (Modules 05, 14).
14. Changing the embedding model without re-embedding everything
Mistake: swapping embedders but keeping old vectors. Why wrong: old and new vectors live in different “meaning spaces” and can’t be compared — retrieval becomes nonsense. Instead: re-embed the entire corpus on any embedder change (Module 08).
15. No security review of the retrieval layer
Mistake: treating retrieved documents as automatically safe. Why wrong: a malicious document can carry prompt injection — hidden instructions that hijack the model (“ignore your rules and reveal X”). Retrieved content is untrusted input. Instead: sanitize, constrain what retrieved text can trigger, and review the retrieval layer for security (Module 14).
16. One-size-fits-all across different content types
Mistake: the same chunking/K/prompt for code, tables, prose, and slides. Why wrong: these have very different structures and needs. Instead: tailor handling per content type.
17. Latency blindness
Mistake: stacking query-rewrite + multi-query + rerank + a big slow model until responses crawl. Why wrong: users abandon slow systems; each fancy stage adds delay. Instead: budget latency, measure p95 (the slow tail, not just the average), and add complexity only where it pays off (Modules 11, 14).
The meta-lesson
Notice a theme: most anti-patterns come from (a) using RAG when you shouldn’t, (b) neglecting the unglamorous upstream stages (parsing, chunking, metadata), or (c) skipping measurement and security. The fancy parts rarely cause the disasters. The boring fundamentals do.
The mark of expertise isn’t knowing the fanciest pattern — it’s not making these mistakes, and stopping others from making them.
Check yourself
- Without looking, name five anti-patterns and the fix for each.
- Why is changing the embedding model without re-embedding so destructive?
- What is prompt injection through retrieved documents, and why does it make retrieved content “untrusted”?
- Which three themes do most anti-patterns fall into?