← Master Index
Vol. 14 Module 14.1 Lecture

Embedding (for retrieval)

RAG Core Concepts

How This Lesson Fits the Module & Volume

After defining RAG, we need the representation that makes “find similar meaning” computable. Embeddings for retrieval turn queries and chunks into vectors ranked by similarity—building on Vol. 12 dense embeddings and contrasting with sparse signals used later in hybrid search.

The next lecture chooses an embedding model; here we focus on what the vectors mean in a RAG pipeline.

Learning Objectives

By the end of this lesson, students should be able to:

  • Define a retrieval embedding as a fixed vector for a query or passage.
  • Explain asymmetric query–document encoding in bi-encoder setups.
  • Encode text with sentence-transformers and rank by cosine similarity.
  • Distinguish embedding use in retrieval vs generative LM token embeddings.
  • List failure modes: domain shift, length bias, and keyword blindness.
  • State why offline document embedding + online query embedding is standard.
Definition

A retrieval embedding is a fixed-length continuous vector produced by an encoder so that texts with related meaning (or matching intent) land near each other under a chosen similarity metric—most often cosine similarity or inner product after L2 normalization.

Retrieval vs LM Token Embeddings

KindUnitRole in RAG
Token / LM tableSubwordInside the generator—not your search index
Passage embeddingChunk / docStored in the vector index
Query embeddingUser questionComputed online; searches the index

Symmetric

  • Same encoder for q & d
  • Good for similar text pairs
  • Common FAQ setups

Asymmetric

  • Query vs long passage
  • May use prefixes / dual towers
  • Typical web/search RAG

Sparse companion

  • BM25 / learned sparse
  • Exact terms & IDs
  • See hybrid later

Code: Query vs Corpus Vectors

import numpy as np from sentence_transformers import SentenceTransformer model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2") docs = ["password reset via email link", "billing cycle ends on the 1st", "SSO with Okta"] query = "I forgot my login password" # Offline: embed & store docs once D = model.encode(docs, normalize_embeddings=True) # Online: embed query only q = model.encode(query, normalize_embeddings=True) scores = D @ q # cosine == dot when L2-normalized order = np.argsort(-scores) for i in order: print(f"{scores[i]:.3f} {docs[i]}")

Strengths

  • Semantic paraphrase match
  • Compact, ANN-friendly vectors
  • Fast online query encode

Tradeoffs

  • Weak on rare exact strings
  • Must re-embed if model changes
  • Chunk text quality dominates
Common Misconception

“Any vector from the chat LLM is fine for search.” Generative models are not automatically trained for cosine retrieval. Use embedding-specialized checkpoints (or fine-tune with contrastive pairs). Mixing embedding spaces across models silently breaks recall.

Knowledge Check

  1. Short Answer: What is stored in the index: query or document embeddings? Answer: Document/chunk embeddings (queries are encoded online).
  2. True/False: Retrieval embeddings are usually sparse one-hots. Answer: False—dense continuous vectors (unless using sparse retrievers).
  3. Multiple Choice: After L2 normalize, cosine equals: (a) Hamming, (b) dot product, (c) edit distance. Answer: (b).
  4. Short Answer: Why encode documents offline? Answer: Avoid re-encoding the corpus on every query; enable ANN search.
  5. True/False: Changing the embedding model requires re-indexing. Answer: True.
  6. Multiple Choice: Bi-encoders primarily support: (a) cheap independent encode + ANN, (b) pixel shaders, (c) SQL joins only. Answer: (a).
  7. Short Answer: Name a failure mode of dense retrieval. Answer: Domain shift, length bias, or missing exact keywords (any).
  8. True/False: Sparse embeddings from Vol. 12 are irrelevant to RAG. Answer: False—they power hybrid search.
  9. Multiple Choice: Next lecture focuses on: (a) Embedding Model, (b) CSS Grid, (c) PCA only. Answer: (a).
  10. Short Answer: What Python library encodes MiniLM passages here? Answer: sentence-transformers.

Key Takeaways

  • Retrieval embeddings map queries and chunks into a shared similarity space.
  • Encode corpus offline; encode queries online.
  • Use retrieval-trained encoders—not arbitrary LM states.
  • Next: Embedding Model.
Trainer’s Guide

Lab: Encode 30 mixed FAQ lines; plot top-3 neighbors for paraphrase vs exact-ID queries.

Prompt: When would asymmetric models outperform a single shared encoder?

Recap: Embeddings make semantic retrieval numeric. Continue with Embedding Model.