Re-ranking — Module 14.1

How This Lesson Fits the Module & Volume

First-stage retrieval (dense, sparse, or hybrid) optimizes for recall. Re-ranking re-scores a shortlist with a stronger model—often a Vol. 12 cross-encoder—so only the best few chunks enter the expensive LLM context. This is Stage 2 precision after Stage 1 recall.

Learning Objectives

By the end of this lesson, students should be able to:

Define re-ranking as second-pass scoring of retrieval candidates.
Contrast bi-encoder ANN with cross-encoder pairwise scoring.
Choose candidate pool size vs final k for latency and quality.
Sketch a cross-encoder re-rank loop in Python.
List lighter alternatives: LLM-as-reranker, heuristic boosts.
Measure lift with nDCG / precision@k before shipping.

Definition

Re-ranking takes an initial ranked list of candidates and produces a new ordering (and usually a shorter top-k) using a more accurate—but costlier—relevance model or heuristics.

Bi-Encoder vs Cross-Encoder

	Bi-encoder	Cross-encoder
Input	Encode q and d separately	Joint [q; d] forward pass
Scale	ANN over millions	Tens–hundreds of pairs
Quality	Good first stage	Usually better pairwise relevance
RAG role	Retrieve candidates	Re-rank before prompt pack

Cross-encoder

Best precision/cost trade
Batch pair scoring
Needs GPU often

LLM re-rank

Flexible criteria
Expensive tokens
Use sparingly

Heuristics

Recency / title boost
Cheap
Limited semantics

Code: Cross-Encoder Re-rank Sketch

from sentence_transformers import CrossEncoder

reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")
query = "What is the refund window?"
candidates = [
    "Refunds are available within 30 days of purchase.",
    "Shipping to EU takes 5–7 business days.",
    "Employees may expense home internet.",
]

pairs = [(query, c) for c in candidates]
scores = reranker.predict(pairs)
order = sorted(range(len(candidates)), key=lambda i: -float(scores[i]))
top = [candidates[i] for i in order[:2]]
print(list(zip([round(float(scores[i]), 3) for i in order], top)))

Strengths

Raises precision of packed context
Cuts noisy chunks that waste tokens
Composable after hybrid fusion

Tradeoffs

Extra latency and compute
Cannot fix empty first-stage recall
Another model to version

Common Misconception

“Re-ranking replaces retrieval.” Cross-encoders cannot scan the whole corpus at query time. If Stage 1 never retrieved the gold chunk, Stage 2 cannot invent it. Fix recall first; then re-rank.

Knowledge Check

Short Answer: What does re-ranking consume as input? Answer: A shortlist of candidates from first-stage retrieval.
True/False: Cross-encoders independently encode query and doc for ANN. Answer: False—they score pairs jointly.
Multiple Choice: Bi-encoders shine at: (a) large-scale ANN, (b) only CSS, (c) PNG encode. Answer: (a).
Short Answer: Why re-rank before the LLM? Answer: Improve precision and reduce noisy context tokens.
True/False: Re-ranking can recover docs never retrieved. Answer: False.
Multiple Choice: Typical candidate pool before re-rank: (a) tens–hundreds, (b) billions pairwise, (c) zero. Answer: (a).
Short Answer: Name a Vol. 12 related lecture. Answer: Cross-encoder (or bi-encoder).
True/False: LLM-as-reranker is always cheaper than MiniLM cross-encoders. Answer: False.
Multiple Choice: Next lecture: (a) Query Expansion, (b) Vol. 1 only, (c) printers. Answer: (a).
Short Answer: Metric to prove re-rank lift? Answer: nDCG, precision@k, or MRR on labeled sets.

Key Takeaways

Re-ranking is precision Stage 2 after recall Stage 1.
Cross-encoders score query–doc pairs on a shortlist.
They cannot fix missing first-stage recall.
Next: Query Expansion.

Trainer’s Guide

Lab: Retrieve top-20 with MiniLM; re-rank to top-5; compare faithfulness of LLM answers.

Prompt: When are heuristic boosts enough without a cross-encoder?

Recap: Re-ranking polishes candidates before generation. Continue with Query Expansion.