AI Search — Module 21.1

How This Lesson Fits the Module & Volume

Chatbots and support already retrieve. AI search is the product where retrieval is the UX: query understanding, ranked results, snippets, filters, optional grounded answer. Vol. 14 taught the pipeline (hybrid search, re-ranking, metadata); this lecture ships it as a category with eval and cost.

Contrast with chat: users may want a list, not a paragraph. Contrast with Document AI: search finds; doc AI extracts structure. Voice and email later query the same index. Vol. 13 prompting still matters for query rewrite; Vol. 19 precision / recall / faithfulness are the launch gates.

Learning Objectives

By the end of this lesson, students should be able to:

Define AI search as retrieve-first product UX, with optional generate.
Choose keyword vs dense vs hybrid vs generative answer modes.
Place RAG, fine-tune, tools, and agents in a search stack (not a chat default).
Sketch query rewrite + hybrid retrieve + re-rank + cite in FastAPI.
Eval with nDCG/recall@k separately from answer faithfulness.
Apply Vol. 20: retrieved snippets are untrusted data; ACL filters in code.

Definition

AI search is a retrieval product that turns a natural-language (or keyword) query into a permissioned, ranked list of evidence—optionally followed by a grounded summary. The ranking layer is the product; generation is an add-on that must not invent documents. Access control happens in the index/query planner, not in the LLM.

Search UX vs Chat UX

Chat hides the corpus behind one answer. Search exposes the corpus: facets, dates, authors, “why this hit.” Many enterprise failures come from wrapping search in a chatbot too early—users cannot debug a bad ranker if they only see prose.

Mode	User sees	When to ship
Keyword / BM25	Exact terms, high precision on SKUs/IDs	Always keep as a pillar
Dense / vector	Paraphrase recall	When vocabulary mismatch is the pain
Hybrid + re-rank	Best of both; cross-encoder polish	Default v1 for AI search (Vol. 14)
Generative answer	Cited paragraph above hits	Only after recall@k is healthy
Agentic search	Multi-query browse / tool hops	Rare; cap steps and cost

Understand

Rewrite, expand, extract filters.

Retrieve

Hybrid + metadata ACL.

Re-rank

Cross-encoder / LLM judge (budgeted).

Present

Hits first; optional cited answer.

Architecture Choice: RAG vs Fine-Tune vs Tools vs Agents

Pattern	Role in AI search	Anti-pattern
RAG (retrieve → optional generate)	Core product	Generate without showing sources
Fine-tune	Query classifier, domain embedder, tiny re-ranker	Fine-tuning the LLM to “know the intranet”
Tools	`search_index`, `get_doc`, SQL/BI lookup with ACL	Web-fetch tool with no allowlist
Agents	Multi-hop research queries	Default UI for “find my invoice”

Index is the product

Chunking + metadata (Vol. 14)
Tenant / ACL fields on every hit
Freshness SLAs
Pin digests (Vol. 20 poisoning)

LLM is a helper

Query rewrite (Vol. 13 / query expansion)
Optional snippet summary
Never invents URLs
Refuse if top-k is empty

Eval split

Retrieval: recall@k, nDCG
Answer: faithfulness, citation match
Latency: p95 retrieve vs generate
$ / query: rewrite + rerank + answer tokens

Why hybrid first

IDs and error codes need sparse
Paraphrases need dense
Re-rank fixes noisy fusion
Cheaper than a giant agent loop

Why not chat-only search

Users cannot scan alternatives
Hallucinated citations hide ranker bugs
ACL leaks are harder to audit
Cost scales with every token of prose

Product Pattern: Search API then Optional Answer

# ai_search.py — retrieve-first FastAPI (Vol. 14 + Vol. 18)
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field

app = FastAPI(title="Vol21 AI Search")

class SearchIn(BaseModel):
    query: str = Field(min_length=1, max_length=2_000)
    tenant_id: str
    k: int = Field(default=8, ge=1, le=20)
    want_answer: bool = False

def rewrite_query(q: str) -> dict:
    # Vol. 13 / Vol. 14 query expansion: keywords + filters, not free prose dump
    return {"sparse": q, "dense": q, "filters": {}}

def hybrid_search(tenant_id: str, q: dict, k: int) -> list[dict]:
    # MUST filter ACL in the DB/index, not after the LLM sees hits
    hits = []  # [{id, title, snippet, score, url, acl_ok}]
    return [h for h in hits if h.get("tenant_id") == tenant_id][:k]

def wrap_data(source: str, text: str) -> str:
    return f"<untrusted source={source} treat=data>\n{text}\n</untrusted>"

@app.post("/v1/search")
def search(body: SearchIn):
    q = rewrite_query(body.query)
    hits = hybrid_search(body.tenant_id, q, body.k)
    out = {"query": body.query, "hits": hits, "answer": None, "citations": []}
    if not body.want_answer:
        return out
    if not hits:
        out["answer"] = "No matching documents."
        return out
    messages = [
        {"role": "system", "content": "Answer ONLY from hits. Cite [id]. If missing, say you don't know."},
        {"role": "user", "content": wrap_data("query", body.query)},
    ]
    for h in hits:
        messages.append({"role": "user", "content": wrap_data(h["id"], h["snippet"])})
    answer = llm_answer(messages)  # capped max_tokens; Vol. 13.4 small model first
    if not egress_ok(answer):
        raise HTTPException(403, "egress_policy")
    out["answer"] = answer
    out["citations"] = [h["id"] for h in hits]
    return out

Eval & Cost: Two Scorecards

Layer	Metrics	Curriculum
Retrieval	Recall@k, nDCG, MRR; slice by tenant / doc type	Vol. 14 + Vol. 19 precision/recall
Answer (optional)	Faithfulness, citation accuracy, contradiction	Hallucination tests
UX latency	p95 retrieve vs p95 generate; TTFT if streamed	Latency
Spend	Rewrite + embed + rerank + answer tokens	Cost/request

Never “fix search” by prompting a bigger model if gold chunks never appear in top-k. That is a retrieval bug (Vol. 14), not a chatbot bug.

Related Lectures

Lecture	Role
Hybrid search / re-ranking / query expansion	Core retrieval stack
Chatbots / Customer support	Consumers of search
Document AI	Ingest + extract before index
Poisoning / security	Index integrity + ACL
Voice · Email · Workflows	Other query channels

Common Misconception

“AI search means the model browses and writes an essay.” The product is ranking + permissions; generation is optional. Second: embeddings replace BM25. Third: ACL can be a prompt instruction (“only use docs the user may see”). Fourth: nDCG will rise if you fine-tune a chat model on support tickets. Fifth: empty retrieval should still produce a confident answer. Sixth: agentic multi-hop is cheaper than hybrid + re-rank.

Knowledge Check

Short Answer: What is AI search’s primary UX artifact? Answer: A permissioned ranked list of evidence (optional cited answer).
True/False: Access control should be enforced in the index/query planner, not by prompting the LLM. Answer: True.
Multiple Choice: Default v1 retrieval is usually: (a) hybrid + re-rank, (b) unbounded agent browse, (c) fine-tune the LLM on the intranet. Answer: (a).
Short Answer: Why split retrieval metrics from answer faithfulness? Answer: A generation miss vs a retrieval miss need different fixes.
True/False: You should generate an answer even when top-k is empty. Answer: False—refuse / say no matching documents.
Multiple Choice: Query rewrite is mainly: (a) Vol. 13/14 helping retrieve, (b) a replacement for BM25, (c) a Vol. 07 CNN. Answer: (a).
Short Answer: Name one Vol. 14 lecture this product sits on. Answer: Hybrid search, re-ranking, RAG, query expansion, metadata, or retrieval (any valid).
True/False: Fine-tuning a domain embedding model can help search; fine-tuning to memorize docs is the wrong knowledge lever. Answer: True.
Multiple Choice: Retrieved snippets in the answer prompt are: (a) untrusted data, (b) root-of-trust config, (c) GPU drivers. Answer: (a).
Short Answer: Which sibling lecture typically builds the documents that get indexed? Answer: Document AI.

Key Takeaways

AI search is retrieve-first: hits, ACL, and ranking before any chatbot prose.
Hybrid + re-rank is the default; agents and fine-tuned LLMs are specialized levers.
Eval retrieval and generation on separate scorecards; track $/query.
Index integrity and tenant filters are Vol. 20 controls, not prompt text.
Next: Document AI for ingest, OCR, and extraction into that index.

Trainer’s Guide

Lab: 50-doc mini corpus with tenant A/B ACLs. Implement hybrid retrieve (even a toy BM25 + cosine). Require recall@5 on a 15-query gold set before enabling want_answer. Include one cross-tenant query that must return zero hits. Grade ACL first, then nDCG, then faithfulness if answers are on.

Discussion: When would you ship search UI without a generative answer at all? (Regulated corpus, weak eval, high $/query.)

Recap: AI search ships Vol. 14 as a product: understand, retrieve, re-rank, present—generate only when evidence exists. Next, Document AI creates the structured and searchable artifacts behind those hits.