Vector Search — Module 14.1

How This Lesson Fits the Module & Volume

A vector database is the store; vector search is the query pattern: encode the question, find nearest chunk vectors, return IDs/text. It specializes retrieval for dense embeddings and leads into the more general idea of similarity search and the concrete metric cosine similarity.

Learning Objectives

By the end of this lesson, students should be able to:

Define vector search as k-NN over embedding space.
Contrast exact vs approximate (ANN) vector search.
Wire query encode → search → hydrate text payloads.
Tune k and score floors for RAG packing.
Explain metric consistency (cosine / IP / L2) with how vectors were trained.
Relate ANN recall/latency knobs to Module 14.1 indexing.

Definition

Vector search finds the k stored vectors most similar to a query vector under a chosen distance or similarity function, returning associated document IDs, scores, and often payloads.

Exact vs Approximate

Mode	How	When
Exact k-NN	Score all vectors	Small N; eval gold standard
ANN	HNSW, IVF, PQ, …	Large N; latency SLOs

Online path

Embed query
ANN search
Fetch chunk text

Offline path

Embed corpus
Build index
Validate recall

RAG pack

Take top-k
Optional re-rank
Fit token budget

Code: Brute-Force Vector Search

import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
ids = ["c1", "c2", "c3"]
texts = ["rotate API keys quarterly", "office Wi-Fi password policy", "travel stipend rules"]
X = model.encode(texts, normalize_embeddings=True)

def vector_search(query: str, k: int = 2):
    q = model.encode(query, normalize_embeddings=True)
    scores = X @ q
    top = np.argsort(-scores)[:k]
    return [{"id": ids[i], "score": float(scores[i]), "text": texts[i]} for i in top]

print(vector_search("how often should we change keys?"))

Strengths

Semantic matching beyond keywords
ANN scales to millions+
Clean API for RAG services

Tradeoffs

ANN can miss true nearest neighbors
Wrong metric tanks quality
Cold-start needs full embed

Common Misconception

“Vector search score = truth.” Similarity scores are relative ranking signals, not calibrated probabilities of correctness. Always validate with labeled queries; consider score thresholds carefully—or re-rank.

Knowledge Check

Short Answer: What does vector search take as input besides k? Answer: A query vector (from embedding the query text).
True/False: ANN always returns the exact nearest neighbors. Answer: False—approximate.
Multiple Choice: Exact search is useful as: (a) eval baseline / small N, (b) only CSS, (c) DNS. Answer: (a).
Short Answer: Why must train metric match index metric? Answer: Embeddings were optimized for a specific similarity/distance.
True/False: Hydrating payloads means fetching chunk text/metadata for IDs. Answer: True.
Multiple Choice: Typical online path starts by: (a) embedding the query, (b) retraining GPT, (c) deleting the index. Answer: (a).
Short Answer: Name an ANN family. Answer: HNSW, IVF, PQ (any).
True/False: High cosine score guarantees a correct RAG answer. Answer: False.
Multiple Choice: Next lecture: (a) Similarity Search, (b) Vol. 1 only, (c) printers. Answer: (a).
Short Answer: How does vector search feed the LLM? Answer: Top hits’ text is packed into the prompt as context.

Key Takeaways

Vector search is k-NN over embeddings for RAG evidence.
Use exact search for small N/eval; ANN for scale.
Scores rank candidates—they are not calibrated truth.
Next: Similarity Search.

Trainer’s Guide

Lab: Compare exact top-10 vs a mocked ANN that drops 20% of neighbors; discuss RAG impact.

Prompt: When would you search with L2 instead of cosine?

Recap: Vector search is how dense RAG finds candidates. Continue with Similarity Search.