Zero-, one-, and few-shot prompting mainly shape what the answer looks like. Chain-of-thought (CoT) shapes how the model reaches it: by eliciting intermediate reasoning steps before the final answer.
CoT is the gateway to Module 13.1’s reasoning cluster—tree-of-thoughts, self-consistency, and reflection all build on the idea that deliberate intermediate text can improve hard tasks.
Learning Objectives
By the end of this lesson, students should be able to:
- Define chain-of-thought prompting and when it helps.
- Write zero-shot CoT cues (e.g., “think step by step”) and few-shot CoT demos.
- Separate scratchpad reasoning from the final answer for parsers and users.
- Identify tasks where CoT helps (math, multi-hop) vs where it adds noise.
- Combine CoT with structured final answers (JSON / labels).
- Preview how sampling multiple CoT paths enables self-consistency.
Chain-of-thought prompting encourages the model to generate explicit intermediate reasoning steps—a verbal “scratchpad”—before committing to a final answer, improving performance on multi-step problems.
Zero-Shot CoT vs Few-Shot CoT
| Style | How you elicit steps | Best for |
|---|---|---|
| Zero-shot CoT | “Let’s think step by step.” / “Show your work.” | Quick wins; capable instruction models |
| Few-shot CoT | Demos that include worked reasoning + answer | Controlling style and rigor of steps |
| Answer-only | No reasoning requested | Easy classification / formatting |
Zero-Shot CoT Prompt
Few-Shot CoT Sketch
Where CoT Helps—and Where It Does Not
Helps
- Arithmetic and symbolic tasks.
- Multi-hop reading comprehension.
- Logic puzzles with constraints.
Often neutral
- Simple sentiment / tagging.
- Pure style rewriting.
- Already one-step lookups.
Risks
- Fluent but wrong reasoning.
- Leaking private scratchpads to users.
- Higher token cost / latency.
Production Pattern: Hidden Scratchpad
Ask the model to reason privately, then emit a machine-readable final field. Downstream systems parse only FINAL (or JSON), while logs retain the chain for debugging. Pair with structured output when answers must be schema-valid.
Strengths and Tradeoffs
Strengths
- Improves multi-step accuracy on many models.
- Debuggable: you can read where it went wrong.
- Composes with few-shot and self-consistency.
Tradeoffs
- Longer outputs cost more.
- Reasoning can be confidently incorrect.
- Not a substitute for tools or retrieval when facts are missing.
“If the chain looks logical, the answer is correct.” Language models can produce coherent-looking justifications for wrong conclusions. Treat CoT as a performance aid and audit trail—not as a proof. For critical answers, verify with tools, tests, or self-consistency.
Knowledge Check
- Short Answer: What does CoT ask the model to produce before the answer? Answer: Explicit intermediate reasoning steps.
- True/False: CoT always improves simple sentiment classification. Answer: False—often neutral or wasteful.
- Multiple Choice: Zero-shot CoT typically uses: (a) gradient updates, (b) a cue like “think step by step,” (c) eight demos minimum. Answer: (b).
- Short Answer: Why separate FINAL from the scratchpad? Answer: Easier parsing / hide reasoning from end users / cleaner APIs.
- True/False: Few-shot CoT demos include worked reasoning, not just answers. Answer: True.
- Multiple Choice: CoT is most associated with gains on: (a) multi-step math/logic, (b) JPEG compression, (c) CSS floats. Answer: (a).
- Short Answer: Name one risk of showing CoT to end users. Answer: Exposing private reasoning, errors, or sensitive intermediate content.
- True/False: Fluent reasoning guarantees correctness. Answer: False.
- Multiple Choice: Next technique that samples many CoT paths: (a) self-consistency, (b) LoRA, (c) pooling. Answer: (a).
- Short Answer: When might answer-only be better than CoT? Answer: Easy one-step tasks where extra tokens do not help.
Key Takeaways
- CoT elicits step-by-step reasoning before the final answer.
- Use zero-shot cues or few-shot worked examples depending on control needs.
- Parse a dedicated final field; do not trust fluent chains as proofs.
- Next: Tree of Thoughts.
Hands-on idea: Same word problem with answer-only vs CoT; have students mark where wrong chains diverge from truth.
Discussion prompt: Should customer-facing apps stream CoT tokens, summarize them, or hide them entirely?
Recap: Chain-of-thought turns hard problems into sequenced intermediate steps. Continue with Tree of Thoughts.