Bias — Module 20.1

How This Lesson Fits the Module & Volume

Vol. 19 taught you to ask does it work?—accuracy, hallucination tests, benchmarks. A model can look SOTA on a public suite and still systematically harm some users. Bias is Vol. 20’s first measurement lens: where error, representation, and opportunity concentrate.

This lecture opens Module 20.1. It is not the same “bias” as Vol. 05 estimation bias (underfitting). Here, bias is a product and data property. Next lecture, Fairness, turns measurements into goals and trade-offs. Later siblings cover explainability, privacy, safety, and governance.

Learning Objectives

By the end of this lesson, students should be able to:

Distinguish statistical estimation bias from representational and allocative bias in AI products.
Name major bias sources: sampling, labels, proxies, historical process, measurement, and aggregation.
Separate bias measurement from fairness goals (this lecture vs the next).
Describe disparate impact as a comparative selection/error-rate concept—not a fake study result.
Slice Vol. 19 metrics by group on a clearly labeled toy table and report gaps without inventing census claims.
Hand off to fairness, explainability, and governance when a gap is found.

Definition

Bias (in this volume) is a systematic difference in how a system represents, scores, or treats people or contexts relative to a stated reference—often visible as unequal error rates, unequal selection rates, stereotyped generations, or missing coverage. It is a measurement and diagnosis problem: you quantify gaps on defined slices. Fairness is the downstream goal and policy choice about which gaps are unacceptable and what to optimize. Vol. 05’s bias is average estimation error; do not mix the two words in a model card without a qualifier.

Two Vocabularies Named “Bias”

Sense	Home lecture	Question it answers
Estimation / inductive bias	Vol. 05 Bias	Is the hypothesis class too simple on average?
Representational bias	This lecture	Whose language, faces, or cases are missing or stereotyped?
Allocative bias	This lecture	Who gets the loan, job screen, or support ticket priority?
Evaluation bias	Vol. 19 + here	Does the benchmark itself under-cover a slice?

A linear model can have low statistical bias and still encode historical hiring patterns. A generative model can ace MMLU-style boards and still produce skewed occupational stereotypes. Vol. 19 quality metrics are necessary; they are not a bias audit.

Where Bias Enters the Pipeline

Source	What happens	Engineer check
Sampling / coverage	Train or eval data miss a dialect, clinic, region, or device.	Slice coverage counts before you slice metrics.
Labeling	Annotators apply uneven standards; noisy or contested labels.	Inter-rater agreement by slice; rubric review (Vol. 04 labeling).
Proxy features	ZIP, name, school, or language stand in for a sensitive attribute.	Ask: would we still use this feature if the proxy were explicit?
Historical process	Past decisions (who was hired, arrested, approved) become labels.	Treat labels as outcomes of a system, not ground truth.
Measurement	The instrument itself is skewed (e.g. one accent ASR worse).	Per-slice error, not only global accuracy.
Aggregation	A “good average” hides a failing subgroup.	Never ship a single headline metric for high-stakes use.

Bias Measurement vs Fairness Goals

This lecture — measure

Define slices (with a legitimate purpose and legal basis to collect attributes—or use proxies only with care).
Compute Vol. 19 metrics per slice: accuracy, F1, selection rate, calibration.
Report gaps, sample sizes, and uncertainty.
Do not invent fake “real-world” demographic studies.

Next lecture — choose

Fairness picks which equality notion is the product goal.
Demographic parity vs equalized odds vs individual similarity.
Trade-offs with utility; impossibility of satisfying every criterion.
Governance: who signs the chosen definition.

Disparate impact (concept)

Compare a rate (selection, error, denial) across groups.
A historical screening heuristic is the “four-fifths” idea: a rate far below the most-favored group’s rate is a flag for investigation, not automatic guilt.
This is educational statistics, not legal advice.

What slicing buys

Catches “good average, bad subgroup” before launch
Gives product and legal a shared numeric language
Connects eval (Vol. 19) to harm, not only leaderboards

What slicing does not buy

A single magic fairness number
Permission to collect sensitive attributes without policy
Proof that the labels themselves are just

Toy Slice Report (Not a Real Demographic Study)

The snippet below uses synthetic group labels G0/G1 on a toy classifier output. It demonstrates how to compute selection rate and error gaps. It is not a census, hiring, or clinical dataset and must not be cited as empirical social science.

# TOY ONLY — synthetic groups, not a real demographic study.
# Do not present these numbers as census, HR, or clinical findings.
import pandas as pd

df = pd.DataFrame({
    "group": ["G0", "G0", "G0", "G0", "G1", "G1", "G1", "G1"],
    "y_true": [1, 0, 1, 0, 1, 0, 1, 0],
    "y_pred": [1, 0, 1, 1, 0, 0, 1, 0],  # invented toy labels
})

def slice_report(frame, pred_positive=1):
    rows = []
    for g, part in frame.groupby("group"):
        n = len(part)
        sel = (part["y_pred"] == pred_positive).mean()
        acc = (part["y_true"] == part["y_pred"]).mean()
        fpr = ((part["y_pred"] == pred_positive) & (part["y_true"] != pred_positive)).sum() / max(
            (part["y_true"] != pred_positive).sum(), 1
        )
        fnr = ((part["y_pred"] != pred_positive) & (part["y_true"] == pred_positive)).sum() / max(
            (part["y_true"] == pred_positive).sum(), 1
        )
        rows.append({"group": g, "n": n, "selection_rate": sel, "accuracy": acc, "fpr": fpr, "fnr": fnr})
    out = pd.DataFrame(rows)
    # Disparate-impact style ratio (concept): min(sel) / max(sel). Tiny-n warning required.
    rates = out["selection_rate"]
    out.attrs["di_ratio"] = float(rates.min() / rates.max()) if rates.max() else None
    return out

report = slice_report(df)
print(report.to_string(index=False))
print("toy di_ratio (min/max selection):", report.attrs["di_ratio"])
print("If n is tiny, do not claim a population effect — widen the sample or withhold the claim.")

Policy companion (also toy): document why you sliced, the legal basis for any sensitive attribute, minimum cell size, and that a ratio is a trigger for review—not a verdict. Pair with transparency (what you publish) and governance (who approves).

Related Lectures

Lecture	Role
Fairness	Turns measured gaps into chosen criteria and trade-offs
Explainability	Why a score differed—without treating SHAP as proof of fairness
Transparency	Model/data cards that disclose known slice limitations
Responsible AI / Governance	Process, owners, and escalation
Benchmarks (Vol. 19)	Why public suites rarely measure harm by group
Estimation bias (Vol. 05)	Same word, statistical meaning

Common Misconception

“If accuracy is high, the model is unbiased.” Global accuracy can hide a failing slice. Second: treating Vol. 05 estimation bias as fairness bias in a stakeholder meeting. Third: publishing a fake or unlabeled demographic “study” from toy rows. Fourth: collecting sensitive attributes “just for the fairness dashboard” without purpose limitation and access control (privacy). Fifth: assuming a four-fifths ratio is a universal legal test rather than one historical investigation heuristic. Sixth: “fixing bias” only by deleting the sensitive column while leaving strong proxies in.

Knowledge Check

Short Answer: In this volume, what is bias primarily—a measurement problem or a single optimization goal? Answer: A measurement/diagnosis problem; fairness is the goal/policy choice.
True/False: Vol. 05 estimation bias is the same concept as representational bias in Vol. 20. Answer: False—same word, different meanings.
Multiple Choice: Allocative bias concerns: (a) who receives a scarce decision/resource, (b) GPU kernel fusion, (c) BLEU n-grams. Answer: (a).
Short Answer: Name two pipeline sources of bias. Answer: Any of: sampling/coverage, labeling, proxies, historical labels, measurement, aggregation.
True/False: A high public benchmark score guarantees low group error gaps. Answer: False—Vol. 19 suites rarely audit harm by slice.
Multiple Choice: Disparate impact (conceptually) compares: (a) selection or error rates across groups, (b) perplexity only, (c) VRAM only. Answer: (a).
Short Answer: Why must the Python example be labeled toy? Answer: So it is not mistaken for a real demographic or clinical study.
True/False: Removing the sensitive attribute always removes bias. Answer: False—proxies can remain.
Multiple Choice: The next lecture after Bias is: (a) Fairness, (b) PCA, (c) ComfyUI. Answer: (a).
Short Answer: What should you report besides a gap ratio? Answer: Slice sizes (n), uncertainty, and that a ratio is a review trigger—not a verdict.

Key Takeaways

Vol. 20 bias is about systematic differences in representation and allocation—not Vol. 05 underfitting.
Measure first (slices, rates, error gaps); choose fairness criteria next.
Disparate impact is a comparative-rate concept and investigation flag, not invented social science.
Never pass off toy tables as real demographic studies; always publish n and purpose.
Next: Fairness — goals, metrics, and trade-offs.

Trainer’s Guide

Lab: Give two toy CSVs (clearly marked synthetic). Students compute per-slice accuracy, FPR/FNR, and selection-rate ratio. Require a written warning if any cell \(n < 30\). Discuss what they would not claim in a blog post.

Whiteboard: Vol. 19 “can it?” → Vol. 20 Bias “for whom is the error?” → Fairness “which equality do we commit to?” Draw proxies as dashed arrows around a deleted sensitive column.

Recap: Bias is how we measure systematic differences after Vol. 19 quality scores look fine. Fairness is the goal we choose next. Use toy slices honestly, treat disparate impact as a comparative flag, and continue to Fairness.