← Master Index
Vol. 14 Module 14.1 Lecture

Cosine Similarity

RAG Core Concepts

How This Lesson Fits the Module & Volume

Under similarity search, cosine similarity is the default score for many RAG embedding models. Understanding the formula clarifies why we L2-normalize, when cosine equals dot product, and how magnitude-insensitive matching behaves. After this metric, Module 14.1 expands to hybrid search.

Learning Objectives

By the end of this lesson, students should be able to:

  • Write the cosine similarity formula and interpret values in [-1, 1].
  • Implement cosine with NumPy and relate it to normalized inner product.
  • Explain why cosine ignores vector length (and when that helps or hurts).
  • Configure indexes for cosine vs IP correctly.
  • Spot numerical issues (zero vectors, float precision).
  • Connect cosine ranking to bi-encoder retrieval from Vol. 12.
Definition

Cosine similarity between vectors a and b is cos θ = (a · b) / (‖a‖ ‖b‖). It measures orientation alignment, not Euclidean magnitude. For L2-normalized vectors, cosine reduces to the inner product a · b.

Properties That Matter for RAG

PropertyImplication
Scale-invariantLonger chunks do not win just by magnitude
BoundedScores in [-1, 1] (often ~[0, 1] for text embeddings)
Equals IP if normalizedPre-normalize for faster ANN IP indexes
Not a probabilityDo not treat 0.82 as “82% true”

Cosine

  • Angle-focused
  • Common for ST
  • Normalize-friendly

Inner product

  • Uses magnitude
  • Some dual encoders
  • Match training!

L2 distance

  • Geometry in R^d
  • Related if unit-norm
  • Index-dependent

Code: NumPy Cosine (+ Normalized Dot)

import numpy as np from sentence_transformers import SentenceTransformer def cosine_similarity(a: np.ndarray, b: np.ndarray) -> float: return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b) + 1e-12)) model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2") a, b, c = model.encode([ "employee parental leave policy", "maternity and paternity leave rules", "how to reset the office printer", ]) print("paraphrase", round(cosine_similarity(a, b), 3)) print("unrelated", round(cosine_similarity(a, c), 3)) an = a / np.linalg.norm(a) bn = b / np.linalg.norm(b) print("normalized dot == cosine?", np.isclose(an @ bn, cosine_similarity(a, b)))

Strengths

  • Robust to embedding scale
  • Interpretable angle semantics
  • Fast after L2 normalize

Tradeoffs

  • Ignores useful magnitude signals
  • Uncalibrated for thresholds
  • Fails on zero / near-zero vectors
Common Misconception

“Cosine 0.9 means the answer is in that chunk.” High cosine means embedding neighborhood, not entailment. A chunk can be topically close yet miss the factual clause you need—hence re-ranking and answer faithfulness checks.

Knowledge Check

  1. Short Answer: Write cosine in words. Answer: Dot product divided by the product of L2 norms.
  2. True/False: For unit vectors, cosine equals the inner product. Answer: True.
  3. Multiple Choice: Cosine is primarily sensitive to: (a) angle, (b) file size, (c) CSS. Answer: (a).
  4. Short Answer: Typical range of cosine? Answer: [-1, 1].
  5. True/False: Cosine scores are calibrated probabilities. Answer: False.
  6. Multiple Choice: Why L2-normalize in RAG indexes? (a) so IP search equals cosine / runs faster, (b) delete docs, (c) train CNNs. Answer: (a).
  7. Short Answer: What happens with a zero vector? Answer: Cosine is undefined / unstable; guard with epsilon or skip.
  8. True/False: High cosine guarantees the chunk answers the question. Answer: False.
  9. Multiple Choice: Next lecture: (a) Hybrid Search, (b) Vol. 1 only, (c) printers. Answer: (a).
  10. Short Answer: Why is scale invariance helpful? Answer: Prevents longer vectors/chunks from dominating by magnitude alone.

Key Takeaways

  • Cosine measures angle; normalize to use fast IP indexes.
  • Scores rank similarity—they are not truth probabilities.
  • Match metric to how the embedding model was trained.
  • Next: Hybrid Search.
Trainer’s Guide

Lab: Compute cosine for paraphrase, contradiction, and keyword-overlap-only pairs; discuss failures.

Prompt: Should support tickets normalize before indexing? Why?

Recap: Cosine similarity is RAG’s default dense ranking score. Continue with Hybrid Search.