Self Consistency — Module 13.1

How This Lesson Fits the Module & Volume

Chain-of-thought produces one reasoning path; tree-of-thoughts searches many deliberately. Self-consistency takes a simpler ensemble idea: sample multiple independent CoT solutions (with temperature > 0) and choose the answer that appears most often.

It is a practical reliability upgrade when you can afford extra generations but do not want a full ToT controller. Pair it with clear final-answer extraction and later with prompt evaluation to decide if the votes are worth the cost.

Learning Objectives

By the end of this lesson, students should be able to:

Define self-consistency and explain why majority vote helps.
Configure sampling (n, temperature) for diverse CoT paths.
Normalize and tally final answers reliably.
Compare self-consistency to ToT and single-path CoT on cost vs accuracy.
Recognize tasks where voting fails (open-ended free text).
Implement a minimal self-consistency loop in application code.

Definition

Self-consistency improves reasoning accuracy by sampling multiple chain-of-thought generations for the same question and selecting the final answer that is most consistent across samples (typically by majority vote).

Method Overview

1. Prompt

CoT + clear FINAL field

2. Sample

n paths, temperature > 0

3. Parse

Extract final answers

4. Vote

Pick majority / mode

Prompt Shape

{{QUESTION}}

Solve the problem carefully. Show your reasoning.
End with exactly one line:
FINAL: <short answer>

Minimal Voting Loop (Pseudocode)

from collections import Counter

def self_consistent_answer(prompt, n=5, temperature=0.7):
    finals = []
    for _ in range(n):
        text = llm.generate(prompt, temperature=temperature)
        finals.append(parse_final(text))   # normalize: strip, lower, unify units
    winner, votes = Counter(finals).most_common(1)[0]
    return winner, votes, finals

Compared with Related Techniques

Technique	Diversity source	Selection	Typical cost
Single CoT	One path	Take it	1×
Self-consistency	i.i.d. samples	Majority vote	n×
Tree-of-thoughts	Structured branches	Heuristic search	Often > n×
Reflection	Critique + revise	Improved draft	~2–3×

When Voting Works

Works well

Math with short numeric answers.
Multiple-choice / discrete labels.
Problems with a unique correct value.

Works poorly

Long open-ended essays.
Many equally valid phrasings.
Tasks needing external tools/facts.

Tuning knobs

n: 5–10 common starting point.
temperature: enough diversity, not chaos.
normalization: critical for fair votes.

Strengths and Tradeoffs

Strengths

Simple to implement; no tree controller.
Often lifts hard reasoning accuracy.
Exposes disagreement as a confidence signal.

Tradeoffs

Linear cost in n.
Majority can still be wrong.
Needs parseable, comparable finals.

Common Misconception

“Self-consistency means the model checks its own work in one pass.” That is closer to reflection. Self-consistency specifically means multiple sampled solutions plus an aggregation rule (vote), not a single self-critique.

Knowledge Check

Short Answer: What does self-consistency aggregate? Answer: Final answers from multiple sampled CoT generations (usually by majority vote).
True/False: Self-consistency typically uses temperature 0 for all samples. Answer: False—diversity needs temperature > 0.
Multiple Choice: Best answer type for voting: (a) long essays, (b) short discrete finals, (c) raw logits only. Answer: (b).
Short Answer: Why normalize answers before voting? Answer: So equivalent forms (e.g., 1/2 vs 0.5) count as the same.
True/False: A unanimous wrong vote is still possible. Answer: True.
Multiple Choice: Relative to ToT, self-consistency is usually: (a) simpler orchestration, (b) always cheaper than 1 CoT, (c) a CNN layer. Answer: (a).
Short Answer: Name a useful side signal from vote counts. Answer: Confidence / disagreement (low vote share ⇒ uncertain).
True/False: Self-consistency is identical to reflection. Answer: False.
Multiple Choice: Cost scales roughly with: (a) number of samples n, (b) image width, (c) kernel size. Answer: (a).
Short Answer: When is self-consistency a poor fit? Answer: Open-ended text with many valid phrasings / no comparable FINAL.

Key Takeaways

Self-consistency = sample many CoT paths, vote on finals.
Needs diversity (temperature) and strict answer parsing.
Great for discrete answers; weak for free-form prose.
Next: Reflection.

Trainer’s Guide

Hands-on idea: Run n=1 vs n=5 on 15 math items; chart accuracy and show a case where the majority is wrong.

Discussion prompt: Should low-agreement answers trigger a tool call or a human review?

Recap: Self-consistency ensembles sampled reasoning paths by majority vote. Continue with Reflection.