Instruction tuning is one important fine-tune. Fine-tuning more generally means continuing training of a pretrained model on task- or domain-specific data so weights specialize. In LLM product work you choose among full FT, PEFT, and no-FT alternatives (prompting/RAG).
Learning Objectives
By the end of this lesson, students should be able to:
- Define fine-tuning and distinguish full FT from PEFT.
- Choose when FT beats prompting/RAG alone.
- List risks: overfitting, catastrophic forgetting, data leakage.
- Describe learning-rate, epoch, and eval hygiene for FT.
- Contrast continued pretraining vs supervised FT.
- Connect FT choices to cost, latency, and governance.
Fine-tuning updates (some or all) parameters of a pretrained model using additional labeled or domain data so the model better matches a target distribution, style, or task—while hopefully retaining useful pretrained skills.
When to Fine-Tune
| Situation | Prefer | Why |
|---|---|---|
| Facts change often | RAG / tools | Weights are a poor live database |
| Stable style / format / jargon | SFT / PEFT | Behavior is sticky in weights |
| Tiny labeled set | Prompting / PEFT | Full FT overfits easily |
| Strict on-prem + fixed schema | FT + eval harness | Deterministic product constraints |
Full Fine-Tune vs PEFT
Full Fine-Tuning
- Updates most/all weights.
- Highest capacity to change.
- Costly; many checkpoints.
PEFT (preview)
- Train small adapters (LoRA…).
- Cheaper multi-tenant adapters.
- Next lecture goes deep.
Continued Pretraining vs SFT
Self-supervised on domain corpus.
Supervised instruction/task pairs.
RLHF/DPO on ranked responses.
Benefits
- Stronger task reliability.
- Can shrink prompt complexity.
- Encodes org-specific workflows.
Risks
- Forgets general skills.
- Memorizes private training text.
- Needs ongoing eval & refresh.
“Fine-tuning always beats a good RAG system.” For knowledge that must stay current or attributable, retrieval usually wins. Fine-tune for behavior and format; retrieve for facts.
Knowledge Check
- Short Answer: What does fine-tuning change? Answer: Model parameters (some or all) using additional training data.
- True/False: Fine-tuning is the only way to specialize an LLM. Answer: False—prompting and RAG often suffice.
- Multiple Choice: For rapidly changing facts, prefer: (a) only full FT, (b) RAG/tools, (c) deleting context, (d) random sampling always. Answer: (b).
- Short Answer: Name one risk of aggressive full FT. Answer: Catastrophic forgetting / overfitting / memorizing secrets.
- True/False: Continued pretraining uses labeled instruction pairs only. Answer: False—it is usually self-supervised on domain text.
- Multiple Choice: PEFT primarily aims to: (a) train fewer extra parameters, (b) remove attention, (c) ban GPUs, (d) skip tokenization. Answer: (a).
- Short Answer: Why use a held-out eval set during FT? Answer: To detect overfitting and regressions on target metrics.
- True/False: Instruction tuning is a form of fine-tuning. Answer: True.
- Multiple Choice: A reason to choose full FT over LoRA is: (a) unlimited free compute always, (b) need large distributional shift capacity, (c) smaller disks magically appear, (d) avoiding all evals. Answer: (b).
- Short Answer: Fine-tune for behavior; retrieve for what? Answer: Facts / up-to-date attributable knowledge.
Key Takeaways
- Fine-tuning specializes pretrained weights to a target distribution.
- Choose FT vs RAG vs prompting based on what must change.
- Full FT is powerful but costly and easy to overfit.
- Next: PEFT makes adaptation parameter-efficient.
Decision clinic: Give three product scenarios; students pick prompt / RAG / PEFT / full FT and justify.
Warning demo: Show a tiny FT run that memorizes training strings verbatim.
Recap: Fine-tuning updates weights to specialize a foundation model—use it deliberately. Continue with PEFT.