Module 13.1 introduced one-shot and few-shot prompting. Here we focus on design quality: which examples to pick, how to format them, and how to avoid teaching the wrong pattern. Good examples are miniature specs.
Learning Objectives
By the end of this lesson, students should be able to:
- Choose diverse, representative examples instead of near-duplicates.
- Format input/output pairs so the mapping is unambiguous.
- Include edge cases and negative examples when needed.
- Avoid leakage of secrets or unstable IDs in demos.
- Decide when zero-shot clarity beats few-shot tokens.
- Align examples with the output format.
Few-shot design is the craft of selecting and formatting a small set of demonstrations that teach the intended input→output mapping—covering typical cases and critical edges without wasting context or biasing the model toward irrelevant surface patterns.
What Makes a Good Example Set
| Property | Good | Bad |
|---|---|---|
| Diversity | Different phrasings & classes | Three near-copies |
| Format fidelity | Matches production schema | Prose demos, JSON required |
| Difficulty | Includes one hard edge | Only trivial cases |
| Label quality | Reviewed gold answers | Noisy or inconsistent labels |
| Safety | Redacted PII | Real customer emails |
Before / After
Before (weak): inconsistent labels and format drift.
After (strong): consistent schema + diverse cases + edge.
Design Patterns
Canonical pair
- Input block
- Output block
- Same delimiters each time
Contrastive pair
- Near-miss input
- Shows wrong vs right
- Great for classifiers
Zero-shot first
- Try clear instructions alone
- Add shots only if eval fails
- Saves tokens
“More shots are always better.” Extra examples can dominate the context, reinforce spurious correlations, and raise cost. Prefer 2–5 high-quality, diverse demonstrations validated by eval—then stop.
Knowledge Check
- Short Answer: What should examples teach? Answer: The intended input→output mapping.
- True/False: Three near-duplicate examples are high-diversity. Answer: False.
- Multiple Choice: Demo format should: (a) match production schema, (b) always be poetry, (c) hide the labels. Answer: (a).
- Short Answer: Why include an edge-case example? Answer: To teach hard boundaries the model might miss.
- True/False: Real PII belongs in few-shot demos. Answer: False.
- Multiple Choice: Contrastive pairs help especially with: (a) CNNs, (b) classifiers / near misses, (c) GPU batching. Answer: (b).
- Short Answer: When might zero-shot beat few-shot? Answer: When clear instructions already pass eval / to save tokens.
- Short Answer: Name one risk of too many shots. Answer: Cost, spurious correlations, or context bloat.
- Multiple Choice: Label quality should be: (a) noisy, (b) reviewed gold, (c) random. Answer: (b).
- True/False: Examples are miniature specifications. Answer: True.
Key Takeaways
- Curate diverse, consistent, redacted demonstrations.
- Match example format to the production output contract.
- Add shots only when eval shows zero-shot is insufficient.
- Use contrastive pairs for tricky boundaries.
- Next: Tone & Style Control.
Hands-on idea: Have teams replace three duplicate shots with one typical, one edge, one contrastive pair and remeasure accuracy.
Discussion prompt: Should few-shot examples live in system or user messages for your product?
Recap: Examples teach by demonstration—design them like tests. Continue with Tone & Style Control.