← Master Index
Vol. 14 Module 14.1 Lecture

RAG Pipeline

RAG Core Concepts

How This Lesson Fits the Module & Volume

Individual pieces—chunking, embeddings, hybrid search, re-ranking, query expansion—only pay off when wired as a RAG pipeline: offline indexing and online serving. This lecture is the assembly diagram before we treat the knowledge base and indexing as first-class topics.

Vol. 13 cost lessons still apply: every stage adds latency and tokens.

Learning Objectives

By the end of this lesson, students should be able to:

  • Draw offline vs online halves of a RAG pipeline.
  • Order stages: ingest → split → embed → index → retrieve → re-rank → generate.
  • Identify observability hooks (latency, recall, faithfulness, cost).
  • Sketch a minimal end-to-end Python pipeline.
  • Plan failure handling when retrieval returns nothing useful.
  • Decide which stages are mandatory vs optional for an MVP.
Definition

A RAG pipeline is the end-to-end system that prepares a knowledge index offline and, online, transforms a user query into retrieved evidence and an LLM answer with logging, budgets, and safety controls.

Two Halves

Offline (build)Online (serve)
Ingest documentsAuth + parse query
Clean / split / chunkOptional expand query
Embed + attach metadataRetrieve (hybrid)
Build ANN / sparse indexesRe-rank + pack prompt
Validate recall smoke testsGenerate + cite + log

MVP

  • Chunk + dense + top-k
  • Simple prompt
  • Basic eval set

Production

  • Hybrid + filters
  • Re-rank + budgets
  • Tracing & alerts

Advanced

  • Multi-query / agents
  • Adaptive k
  • Online learning loops

Code: End-to-End Mini Pipeline

from sentence_transformers import SentenceTransformer, util # --- offline --- model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2") docs = ["Refunds within 30 days.", "EU shipping 5–7 days.", "Support 9–6 IST weekdays."] doc_emb = model.encode(docs, normalize_embeddings=True, convert_to_tensor=True) # --- online --- def rag_answer(question: str, k: int = 2) -> str: q_emb = model.encode(question, normalize_embeddings=True, convert_to_tensor=True) hits = util.semantic_search(q_emb, doc_emb, top_k=k)[0] if not hits or hits[0]["score"] < 0.25: return "I don't have enough evidence in the knowledge base." ctx = "\n".join(f"[{i}] {docs[h['corpus_id']]}" for i, h in enumerate(hits)) prompt = f"Answer using only context. Cite [#].\n{ctx}\n\nQ: {question}\nA:" return prompt # swap for LLM client.generate(prompt) print(rag_answer("How long for EU delivery?"))

Strengths of a clear pipeline

  • Debuggable stage boundaries
  • Swappable components
  • Measurable SLOs per stage

Tradeoffs

  • More moving parts than chatbots
  • Index freshness jobs required
  • Cost stacks across stages
Common Misconception

“The pipeline is just prompt + vector search.” Production RAG includes ingest, versioning, ACL filters, eval gates, token budgets, refusal behavior, and observability. Skipping those turns demos into incidents.

Knowledge Check

  1. Short Answer: Name the two pipeline halves. Answer: Offline index build and online query serving.
  2. True/False: Re-ranking belongs in the offline half only. Answer: False—it runs online on candidates.
  3. Multiple Choice: MVP usually includes: (a) chunk+embed+retrieve+generate, (b) only Kubernetes, (c) printers. Answer: (a).
  4. Short Answer: What should happen on low retrieval scores? Answer: Refuse / say insufficient evidence rather than invent.
  5. True/False: Vol. 13 token budgets still matter in RAG. Answer: True.
  6. Multiple Choice: Observability should track: (a) latency, recall, cost, faithfulness, (b) only font size, (c) DPI. Answer: (a).
  7. Short Answer: Why separate stages? Answer: Swap, debug, and measure components independently.
  8. True/False: Index freshness is optional in changing corpora. Answer: False.
  9. Multiple Choice: Next lecture: (a) Knowledge Base, (b) Vol. 1 only, (c) CSS. Answer: (a).
  10. Short Answer: Online stage after retrieve, before generate? Answer: Re-rank and/or prompt packing (also filters).

Key Takeaways

  • RAG pipelines split offline indexing from online serve.
  • Compose Module 14.1 stages deliberately; start MVP then harden.
  • Instrument quality, latency, and cost at each boundary.
  • Next: Knowledge Base.
Trainer’s Guide

Lab: Whiteboard a pipeline for an HR handbook bot; mark which stages are MVP vs week-2.

Prompt: Where would you put PII redaction—ingest, retrieve, or generate?

Recap: The RAG pipeline assembles Module 14.1 into a shippable system. Continue with Knowledge Base.