Alignment wants honesty; hallucination is when a model produces fluent, plausible content that is not grounded in facts or provided context. It is not a bug in sampling alone—it follows from generative modeling. Mitigation is a systems problem: RAG, tools, abstention, and evals—not wishful thinking.
Learning Objectives
By the end of this lesson, students should be able to:
- Define hallucination in LLM outputs.
- Distinguish intrinsic vs extrinsic (context-grounded) errors.
- Explain why next-token models invent plausible details.
- List mitigation patterns: RAG, citations, constrained decoding, abstain.
- Design evals that catch factual and groundedness failures.
- Connect hallucinations to overlong or noisy context.
An LLM hallucination is generated content that is presented as factual or faithful but is incorrect, fabricated, or unsupported by the available grounding sources (training knowledge as used, retrieved docs, or user-provided context).
Types Practitioners Care About
| Type | Meaning | Example |
|---|---|---|
| Factual | Wrong world knowledge | Invented citation or date |
| Faithfulness | Contradicts given context | Summary adds claims absent from doc |
| Attribution | Wrong source linkage | Cites Doc A for a Doc B fact |
Why It Happens
Model
- Trained to continue fluently.
- No built-in truth oracle.
- Calibration often poor.
System
- Missing retrieval.
- Bad prompts (“be confident”).
- No abstention path.
Mitigation Stack
RAG / tools / DB queries.
Schemas, quotes, cite spans.
Check claims vs sources.
Say “I don’t know” when unsure.
What Helps
- High-quality retrieved context.
- Force citations + span checks.
- Lower temperature for factual tasks.
What Doesn’t Suffice
- “Please don’t hallucinate” alone.
- Bigger models without grounding.
- Blind trust in fluent prose.
“Hallucinations mean the model is broken.” Fluent invention is expected under a generative objective. Production systems must assume it and design for grounding and verification.
Knowledge Check
- Short Answer: What is an LLM hallucination? Answer: Fluent content that is false or ungrounded relative to facts/context.
- True/False: Next-token training alone guarantees factual answers. Answer: False.
- Multiple Choice: A faithfulness error is: (a) contradicting provided context, (b) perfect citation, (c) faster GPUs, (d) smaller vocabularies. Answer: (a).
- Short Answer: Name two mitigation strategies. Answer: RAG/tools, citation checks, abstention, constrained decoding, verification...
- True/False: Telling the model not to hallucinate fully solves the problem. Answer: False.
- Multiple Choice: For factual Q&A, a strong pattern is: (a) no context ever, (b) retrieve-then-generate with verification, (c) maximize temperature always, (d) delete the system prompt. Answer: (b).
- Short Answer: Why can summaries hallucinate? Answer: The model may add plausible details not present in the source.
- True/False: Lower temperature can reduce some creative fabrications on closed tasks. Answer: True—though it is not a complete fix.
- Multiple Choice: Attribution hallucination means: (a) wrong source linkage, (b) perfect retrieval, (c) no tokens, (d) only vision inputs. Answer: (a).
- Short Answer: What should a model do when evidence is missing? Answer: Abstain or ask for more context rather than invent.
Key Takeaways
- Hallucination is fluent ungrounded generation.
- Treat it as expected; mitigate with systems design.
- Ground, constrain, verify, abstain.
- Next: Context Length—how much text the model can condition on.
Demo: Ask for a niche paper citation; show fabrications; then redo with a provided abstract and require quotes.
Metric: Build a 20-item faithfulness quiz with gold source docs.
Recap: Hallucinations are a structural LLM risk; grounding and verification are mandatory. Continue with Context Length.