← Master Index
Vol. 14 Module 14.1 Lecture

Embedding Model

RAG Core Concepts

How This Lesson Fits the Module & Volume

You know embeddings for retrieval; now pick the model that produces them. Choices trade dimension, latency, domain fit, and license—echoing Vol. 12’s bi-encoder vs cross-encoder split: bi-encoders fill the index; cross-encoders re-rank later.

After the model is fixed, Module 14.1 moves to how text is cut into chunks.

Learning Objectives

By the end of this lesson, students should be able to:

  • Define an embedding model as an encoder trained for similarity / retrieval.
  • Compare open local models vs hosted embedding APIs on ops axes.
  • Select dimension, max sequence length, and normalize settings deliberately.
  • Explain why one embedding space must be consistent across the corpus.
  • Describe when to fine-tune vs swap a stronger off-the-shelf checkpoint.
  • Position bi-encoders for indexing and cross-encoders for re-ranking.
Definition

An embedding model is a neural encoder (often a Transformer bi-encoder) trained so that related texts have high vector similarity. In RAG it is the function f(text) → Rd used for both corpus indexing and query encoding.

Selection Checklist

AxisAskWhy it matters
DomainGeneral web vs legal/code/medical?Out-of-domain recall collapses
Dim / size384 vs 768+?Index RAM, ANN speed, quality
Context lengthChunk size fit?Truncation destroys meaning
Latency / costLocal GPU vs API?CPSR and SLOs
License / PIICan text leave the VPC?Compliance

Local ST model

  • Full control
  • One-time infra
  • You own versioning

Hosted API

  • Fast to start
  • Per-token / call cost
  • Vendor lock-in risk

Fine-tuned

  • Best domain fit
  • Needs labeled pairs
  • Re-index on deploy

Code: Load, Encode, Version

from sentence_transformers import SentenceTransformer MODEL_ID = "sentence-transformers/all-MiniLM-L6-v2" # pin in config + index metadata model = SentenceTransformer(MODEL_ID) texts = ["Reset MFA from the security settings page."] vectors = model.encode( texts, normalize_embeddings=True, # match your index metric batch_size=64, show_progress_bar=False, ) print(MODEL_ID, vectors.shape, vectors.dtype) # Persist MODEL_ID with every index build so query-time encode matches.

Strengths of a pinned model

  • Reproducible retrieval
  • Clear upgrade path
  • Evalable A/B swaps

Tradeoffs

  • Upgrades force full re-embed
  • Larger models cost more
  • Wrong domain wastes ANN quality
Common Misconception

“Bigger embedding models always win RAG evals.” Extra dimensions help until they do not—noise, latency, and RAM can erase gains. Measure recall@k and answer faithfulness on your corpus; prefer the smallest model that hits the SLO.

Knowledge Check

  1. Short Answer: What must stay consistent between index build and query time? Answer: The same embedding model (and normalize/metric settings).
  2. True/False: Cross-encoders are ideal for embedding millions of docs for ANN. Answer: False—too slow; use bi-encoders for indexing.
  3. Multiple Choice: Pinning MODEL_ID in index metadata helps: (a) fashion, (b) reproducibility / safe upgrades, (c) CSS. Answer: (b).
  4. Short Answer: Name two selection axes. Answer: Domain, dim, latency, license, context length (any two).
  5. True/False: Mixing two embedding APIs in one index is fine. Answer: False—spaces are incompatible.
  6. Multiple Choice: Fine-tuning mainly needs: (a) similarity pairs/labels, (b) only CSS, (c) printers. Answer: (a).
  7. Short Answer: Why normalize embeddings? Answer: So cosine equals dot product and metrics match the index.
  8. True/False: Hosted APIs never affect CPSR. Answer: False—they add per-call cost.
  9. Multiple Choice: After choosing a model, Module 14.1 covers: (a) Chunking, (b) Vol. 1 only, (c) audio codecs. Answer: (a).
  10. Short Answer: Role of bi-encoders vs cross-encoders? Answer: Bi-encode for index/search; cross-encode to re-rank small sets.

Key Takeaways

  • Pick embedding models for domain, dim, latency, and compliance—not hype.
  • Pin model IDs; never mix spaces in one index.
  • Bi-encoders index; cross-encoders re-rank.
  • Next: Chunking.
Trainer’s Guide

Lab: Compare MiniLM vs a larger ST model on 40 labeled query–doc pairs; plot recall@5 vs encode ms.

Discussion: Write the runbook for swapping embedding models in production.

Recap: The embedding model is a versioned contract with your index. Continue with Chunking.