Few Shot — Module 13.1

How This Lesson Fits the Module & Volume

One-shot shows a single pattern. Few-shot supplies several demonstrations so the model can infer a richer mapping—class boundaries, edge cases, and consistent formatting—still without updating weights.

Few-shot is the workhorse of Module 13.1 before reasoning techniques. It pairs later with chain-of-thought (few-shot CoT) and with structured formats in JSON prompting. Always weigh token cost and demo curation effort against measured gains over zero-shot.

Learning Objectives

By the end of this lesson, students should be able to:

Define few-shot prompting and choose an appropriate number of examples.
Curate a diverse, non-redundant demo set that covers key classes and edges.
Order and format demonstrations to maximize pattern clarity.
Measure few-shot gains against zero-shot and one-shot baselines.
Detect demo bias, label imbalance, and answer leakage.
Know when few-shot should yield to fine-tuning / adapters from Volume 12.

Definition

Few-shot prompting places a small number (typically 2–8) of input–output demonstrations in the context window, then asks the model to continue the same mapping for a new input.

How Many Shots?

Shots	Typical use	Watch out
2–3	Format + a couple of classes	May miss rare edges
4–6	Multi-class labeling, extraction	Token cost rises fast
8+	Complex idiosyncratic schemas	Diminishing returns; context pressure

There is no universal optimum. Increase shots until the eval plateaus or the context budget hurts latency and cost.

Few-Shot Template

Extract {person, role, company} as JSON. Use null if missing.

Example 1
Input: Ada joined as CTO of Contoso last May.
Output: {"person":"Ada","role":"CTO","company":"Contoso"}

Example 2
Input: We spoke with Sam about the partnership.
Output: {"person":"Sam","role":null,"company":null}

Example 3
Input: Priya, a staff engineer at Globex, led the review.
Output: {"person":"Priya","role":"staff engineer","company":"Globex"}

Input: {{NEW_SENTENCE}}
Output:

Curation Rules

Cover

Each important class at least once.
One empty / null case.
One hard but realistic edge.

Avoid

Near-duplicate demos.
All-positive or all-easy sets.
Contradictory labels across demos.

Order

Keep schema identical every time.
Put clearest pattern early.
End with a typical case, not a freak outlier.

Few-Shot vs PEFT

If you need the same idiosyncratic behavior on millions of calls, demos burned into every prompt become expensive. That is when Volume 12 tools—LoRA, adapters, SFT—encode the mapping into parameters and leave prompts shorter.

Strengths

No training; rapid iteration.
Transparent: demos are auditable.
Great for schemas and niche labels.

Tradeoffs

Tokens scale with shot count.
Sensitive to demo choice and order.
Harder to maintain as product rules change.

Common Misconception

“More shots always help.” Extra demos can add noise, imbalance, or contradictory cues. After a quality plateau, more shots mainly buy latency and cost—measure with a held-out set (see prompt evaluation).

Knowledge Check

Short Answer: What is few-shot prompting? Answer: Including several input–output demos in the prompt before the new query.
True/False: Few-shot updates model weights. Answer: False.
Multiple Choice: A good few-shot set usually includes: (a) only easy duplicates, (b) diverse classes and an edge/null case, (c) the test answers. Answer: (b).
Short Answer: Why might 8+ shots hurt? Answer: Context cost, latency, diminishing returns, possible noise.
True/False: Contradictory labels across demos confuse the model. Answer: True.
Multiple Choice: Prefer PEFT over perpetual few-shot when: (a) one-off script, (b) high-volume durable specialty, (c) you dislike JSON. Answer: (b).
Short Answer: Name one curation anti-pattern. Answer: Near-duplicates / all-easy / leakage / imbalance (any).
True/False: Few-shot can be combined with chain-of-thought. Answer: True (few-shot CoT).
Multiple Choice: Baseline before claiming few-shot wins: (a) zero-shot, (b) random strings, (c) CNNs. Answer: (a).
Short Answer: What should stay identical across demos? Answer: The output schema / formatting pattern.

Key Takeaways

Few-shot teaches mappings by multiple demonstrations in context.
Curate for coverage and consistency; do not assume more is better.
Escalate to PEFT when the pattern is stable and high-volume.
Next: Chain of Thought.

Trainer’s Guide

Hands-on idea: Ablate 0/1/3/6 shots on a 50-item extract task; plot accuracy vs tokens.

Discussion prompt: How would you version-control a living few-shot library as product rules change?

Recap: Few-shot uses a small demo set to teach complex mappings at inference time. Continue with Chain of Thought.