Privacy — Module 20.1

How This Lesson Fits the Module & Volume

Transparency says what you disclose. Privacy limits what you may collect, keep, embed, retrieve, and log. Vol. 04 data leakage was about future/forbidden features inflating offline metrics. Here, leakage is often personal or confidential information appearing where it should not—in prompts, RAG answers, fine-tunes, screenshots, or vendor logs.

Privacy sits beside security, AI safety, and compliance. Fairness dashboards that collect sensitive attributes (bias) are still privacy systems. This lecture is engineering hygiene and policy patterns—not a full legal opinion on any jurisdiction.

Learning Objectives

By the end of this lesson, students should be able to:

Define PII/personal data at a working engineer level and give examples in prompts and tickets.
Apply minimization, purpose limitation, and retention to training, logs, and RAG corpora.
Relate Vol. 04 train/test leakage to RAG/prompt leakage conceptually (information in the wrong place).
List RAG-specific leak paths: over-retrieval, citation dump, embedding store, vendor logging.
Implement a small defensive redaction/allowlist sketch and a retention policy stub.
Know when privacy conflicts with explainability/transparency and how to resolve via governance.

Definition

Privacy (in this module) is control over personal and confidential information across the AI lifecycle: collection, training, retrieval, generation, logging, sharing, and deletion. PII / personal data includes identifiers and data that can reasonably identify a person (name, email, phone, ID numbers, precise location, account IDs—and often free text that contains them). Minimization means collect only what the purpose needs. Retention means delete or de-identify when the purpose ends. RAG leakage is when a retriever or generator exposes corpus text (or embeddings that reconstruct it) to unauthorized users or to the model vendor. Vol. 04 leakage is a cousin: both are “information used outside its allowed context.”

Vol. 04 Leakage vs Privacy Leakage

	Vol. 04 data leakage	This lecture (privacy / RAG)
Wrong place	Test/future info in training or features	Personal/confidential text in logs, answers, or vendor APIs
Symptom	Inflated offline metrics, prod crash	User harm, contract breach, regulatory exposure
Typical fix	Honest splits, pipelines, no target proxies	Minimization, redaction, ACLs, retention, no-train flags
Eval cousin	Vol. 19 validity of scores	Canary docs, access tests, red-team retrieval

Where Personal Data Shows Up in LLM Products

Collection & prompts

Users paste emails, IDs, medical notes.
Support tickets and CRM fields.
Screen recordings and eval traces.

RAG & embeddings

Chunk stores indexed without ACLs.
Answer quotes a private doc to the wrong user.
Metadata (author, path) more sensitive than the chunk.

Training & vendors

Fine-tunes on production chats by accident.
Provider logs prompts unless contract/UI says otherwise.
“Improve the model” toggles left on.

Minimization + retention buy

Smaller blast radius after a breach
Cleaner data cards (transparency)
Fewer impossible deletion requests

Skipping them costs

PII in embeddings you cannot easily unlearn
Fairness slice stores without purpose limitation
Explanations that echo secrets back to chat

Defensive Redaction + Retention Stub

The patterns below are defensive hygiene for systems you operate: reduce what you store and what the model can repeat. They are not a complete privacy program and not instructions to collect or monitor other people’s devices.

import re
from datetime import datetime, timedelta, timezone

# Defensive minimization on text YOU already received in your app.
EMAIL = re.compile(r"\b[\w.+-]+@[\w-]+\.[\w.-]+\b")
PHONE = re.compile(r"\b(?:\+?\d{1,3}[-. ]?)?(?:\(?\d{3}\)?[-. ]?)\d{3}[-. ]?\d{4}\b")

def redact_pii(text: str) -> str:
    text = EMAIL.sub("[EMAIL]", text)
    text = PHONE.sub("[PHONE]", text)
    return text

def rag_acl_ok(user_id: str, chunk_meta: dict) -> bool:
    """Only retrieve chunks the caller is allowed to see (privacy = access control)."""
    allowed = set(chunk_meta.get("allowed_users", [])) | set(chunk_meta.get("allowed_roles", []))
    return user_id in allowed or "public" in chunk_meta.get("allowed_roles", [])

# Retention policy stub (pair with real deletion jobs + vendor no-train flags):
POLICY = {
    "purpose": "support ticket routing only",
    "prompt_logs_days": 90,
    "embeddings_follow_source_doc_ttl": True,
    "fine_tune_on_prod_chats": False,
    "vendor_opt_out_training": True,
    "canary_doc_ids": ["privacy-canary-001"],  # must never appear in another tenant's answer
}

def expired(ts: datetime, days: int) -> bool:
    return datetime.now(timezone.utc) - ts > timedelta(days=days)

# Eval idea (conceptual): inject a unique canary sentence into tenant A's corpus.
# Fail the build if tenant B's RAG answer contains that canary.
# Related conceptually to Vol. 04 leakage: information crossing a boundary it should not.

Policy companion: purpose limitation for any demographic attributes used in fairness eval; separate production ACL from research sandbox; document retention on the data card. Differential privacy and formal anonymization are advanced controls—mention them as options, do not claim a regex is anonymization.

Related Lectures

Lecture	Role
Data leakage (Vol. 04)	Eval/feature leakage cousin: information in the wrong context
Transparency	What you may say vs what you must not publish
Explainability	Attributions and citations can reveal PII
Bias / Fairness	Slice attributes are still personal data
Security / Prompt injection	Exfiltration via prompts and tools
Compliance	Jurisdiction-specific duties (high level)

Common Misconception

“We do not store PII; we only log prompts.” Prompts are often PII. Second: embeddings are harmless because they are vectors—they can still memorize or retrieve secrets. Third: Vol. 04 leakage and privacy leakage are unrelated. Fourth: regex redaction equals anonymization. Fifth: turning on provider “improve the model” without a contract review. Sixth: dropping document ACLs because “the LLM will only answer relevant questions.”

Knowledge Check

Short Answer: What is minimization? Answer: Collect/keep only the personal data needed for a stated purpose.
True/False: Prompt logs cannot contain PII. Answer: False—they often do.
Multiple Choice: RAG leakage is when: (a) unauthorized users or tenants see corpus text via retrieval/generation, (b) BLEU is low, (c) the GPU is warm. Answer: (a).
Short Answer: How is Vol. 04 data leakage conceptually related? Answer: Both are information used outside its allowed context (eval vs personal/confidential exposure).
True/False: Regex substitution is full anonymization. Answer: False—it is a partial defensive redaction.
Multiple Choice: Retention means: (a) delete or de-identify when the purpose ends, (b) keep forever for vibes, (c) publish the data card samples. Answer: (a).
Short Answer: Name one RAG leak path besides the generated answer. Answer: Any of: embedding store, metadata, vendor logs, citations dumping private text.
True/False: Fairness slice attributes are exempt from privacy rules. Answer: False—they still need purpose limitation and access control.
Multiple Choice: A canary document in tenant A appearing in tenant B’s answer is: (a) a privacy/ACL eval failure, (b) a good benchmark, (c) mixed precision. Answer: (a).
Short Answer: Why mention vendor no-train / opt-out flags? Answer: So prompts are not used to train third-party models without agreement.

Key Takeaways

Privacy = purpose, minimization, access, retention, and deletion across prompts, RAG, training, and logs.
Vol. 04 leakage and RAG/PII leakage share a boundary idea: information in the wrong context.
Embeddings and citations can leak; ACLs belong in retrieval, not only in the UI.
Redaction helps; it is not anonymization or a full compliance program.
Next: AI Safety — misuse, overreliance, capability vs control.

Trainer’s Guide

Lab: Students map data flows for a toy RAG bot (user → prompt log → embed → retrieve → vendor API). Mark PII, TTLs, and ACL checkpoints. Add a canary-doc test description (no live attack on third-party systems).

Whiteboard: Vol. 04 split leakage vs tenant-isolation leakage. Draw “explain this answer” accidentally echoing a Social Security number.

Recap: Privacy is minimization, retention, and access control for personal data in AI systems—including RAG leakage, a cousin of Vol. 04 leakage. Continue to AI Safety.