Retrieval — Module 14.1

How This Lesson Fits the Module & Volume

Chunks exist; now the live system must retrieve the right ones for a query. Retrieval is the quality bottleneck of RAG: the generator cannot cite what never entered the prompt. This lecture frames the stage; later ones detail vector databases, vector search, hybrid search, and re-ranking.

Learning Objectives

By the end of this lesson, students should be able to:

Define retrieval as selecting top-k evidence units for a query.
Contrast sparse, dense, and hybrid retrieval at a high level.
Choose k and score thresholds with quality and token cost in mind.
Describe recall-oriented first stage vs precision-oriented second stage.
List evaluation metrics: recall@k, MRR, nDCG, and grounded-answer rate.
Explain why “retrieve then generate” fails if filters exclude the gold doc.

Definition

Retrieval is the process of ranking and returning the most relevant items from a knowledge store for a query—typically returning top-k chunks with scores (and metadata) for downstream prompting or re-ranking.

Retrieval Families

Family	Signal	Vol. 12 link
Sparse	Term overlap / BM25 / learned sparse	Sparse embeddings
Dense	Embedding similarity	Dense embeddings
Hybrid	Fuse sparse + dense	Hybrid embeddings

Stage 1

High recall, cheap
ANN / BM25
k = 20–100

Stage 2

High precision
Cross-encoder re-rank
k′ = 3–10 for LLM

Filters

Metadata / ACL
Time / tenant
Apply carefully

Code: Top-k Dense Retrieval

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
corpus = ["VPN setup guide", "Expense policy FY26", "Onboarding checklist"]
meta = [{"id": "d1"}, {"id": "d2"}, {"id": "d3"}]
corpus_emb = model.encode(corpus, normalize_embeddings=True, convert_to_tensor=True)

def retrieve(query: str, k: int = 2):
    q = model.encode(query, normalize_embeddings=True, convert_to_tensor=True)
    hits = util.semantic_search(q, corpus_emb, top_k=k)[0]
    return [
        {"text": corpus[h["corpus_id"]], "score": float(h["score"]), **meta[h["corpus_id"]]}
        for h in hits
    ]

print(retrieve("how do I connect remotely?"))

Strengths of explicit retrieval

Auditable evidence path
Separates search from generation
Enables hybrid / re-rank upgrades

Tradeoffs

Misses cascade to wrong answers
k too high burns tokens
Needs offline + online eval

Common Misconception

“If the LLM is strong, weak retrieval is fine.” A powerful generator with empty or wrong context still invents. Invest in retrieval eval (recall@k on labeled questions) before prompt polish.

Knowledge Check

Short Answer: What does retrieval return to the RAG prompt? Answer: Top-k relevant chunks (with scores/metadata).
True/False: Dense retrieval uses term-frequency alone. Answer: False.
Multiple Choice: Stage-1 goal is usually: (a) max precision only, (b) high recall cheaply, (c) CSS. Answer: (b).
Short Answer: Name one retrieval metric. Answer: recall@k, MRR, or nDCG.
True/False: Raising k always improves answer quality. Answer: False—noise and cost can hurt.
Multiple Choice: Cross-encoders typically appear in: (a) stage-2 re-rank, (b) PNG export, (c) DNS. Answer: (a).
Short Answer: Why can metadata filters hurt? Answer: Over-filtering can drop the gold document.
True/False: Hybrid retrieval combines sparse and dense signals. Answer: True.
Multiple Choice: Next lecture: (a) Vector Database, (b) Vol. 1 only, (c) printers. Answer: (a).
Short Answer: Why is retrieval the RAG bottleneck? Answer: The model can only use evidence that was retrieved.

Key Takeaways

Retrieval selects the evidence that grounds generation.
Use cheap high-recall first stage; optional precise re-rank.
Evaluate recall@k—do not only eyeball answers.
Next: Vector Database.

Trainer’s Guide

Lab: Build a 100-chunk corpus; measure recall@5 before/after adding a synonym-heavy query set.

Discussion: Who owns retrieval SLOs vs generation SLOs on your team?

Recap: Retrieval is the gatekeeper of grounded answers. Continue with Vector Database.