Governance — Module 20.1

How This Lesson Fits the Module & Volume

Responsible AI defined the operating loop. Governance is who owns it: roles, review boards, risk registers, and a model inventory so Vol. 13 prompts, Vol. 14 indexes, Vol. 15 agents, and fine-tunes are not invisible shadow IT. Without governance, eval gates have no enforcer and incidents have no commander.

This lecture is organizational design for AI products—not a substitute for corporate law. Compliance next translates high-level regulatory regimes into engineering implications. The volume capstone, security, threat-models the whole LLM app.

Learning Objectives

By the end of this lesson, students should be able to:

Assign RACI-style roles for AI product, safety, security, legal, and ops.
Describe what an AI review board decides (go/no-go, residual risk).
Maintain a model / system inventory (artifacts, owners, data, tools).
Keep a living risk register linked to threat classes already taught.
Connect change control (prompt, index, adapter, tools) to review thresholds.
See governance artifacts as inputs to compliance and security capstone.

Definition

AI governance is the set of roles, forums, and records that decide which AI systems may run, under what risk appetite, with which controls, and who is accountable when they fail. It turns responsible-AI practices into durable institutional memory: inventories, risk registers, board minutes, and change tickets. Governance does not replace engineering; it makes engineering auditable and interruptible.

Roles (RACI Sketch)

Role	Accountable for	Typical decisions
Product owner	Use case, UX, residual product risk	Ship vs delay; scope cuts
AI / ML lead	Model, prompts, RAG quality	Eval metrics; rollback of adapters/indexes
Safety owner	Policy categories, false refusals	Safety gate thresholds
Security owner	Injection, jailbreak, poisoning, access	Tool allowlists; integrity gates
Privacy / legal (advisory)	Data uses, notices (not this course’s legal advice)	Escalate to counsel; block high-risk uses
Ops / SRE	Runtime, logging, on-call	Containment switches
Review board	Cross-functional go/no-go	Accept residual risk or require controls

Small teams still name humans, even if one person wears two hats. Anonymous “the model” is not an owner. Vol. 15 HITL reviewers are a runtime role; they need a manager and a queue SLO, not only a checkbox in a design doc.

Review Board

In scope

New high-impact use cases (HR, credit, medical adjacent, minors)
New tools with side effects
Training / fine-tune / major index refresh
Sev1/2 incident closure and residual risk

Inputs

System card / inventory row
Eval gate results (Vol. 19 + safety/fairness)
Threat model (injection / tools / data)
Risk register deltas

Outputs

Approve / approve-with-conditions / reject
Required controls (HITL, logging, DPIA-like notes)
Revisit date
Minutes (auditable)

Intake

Inventory + risk draft.

Review

Board or lightweight path.

Conditions

Controls before prod.

Revisit

After drift or incident.

Model Inventory & Risk Register

If you cannot list the systems, you cannot govern them. An inventory row is a system, not only a foundation-model name: prompts, retrieval corpora, tools, eval suites, and owners. The risk register is the living list of “what could hurt people or the business,” scored and linked to controls from prior lectures.

Artifact	Minimum fields
Inventory	ID, purpose, users, model IDs + digests, prompt refs, indexes, tools, data classes, owners, status (dev/stage/prod)
Risk register	ID, threat class, scenario (plain language), likelihood, impact, controls, residual risk, owner, review date
Change ticket	What changed (prompt/index/adapter/tool), eval diff, board path (full vs expedited)

Defensive Snippet: Inventory + Risk Register

# Governance records as structured data. Not legal advice.

from datetime import date

INVENTORY = {
    "sys_id": "AI-HELP-004",
    "name": "Support copilot",
    "purpose": "Draft replies from approved KB only",
    "users": "internal_support_agents",
    "model": {"provider": "vendor-x", "id": "chat-large", "digest": "sha256:abc..."},
    "prompts": ["prompts/support_system_v3.md"],
    "indexes": [{"name": "kb_prod", "manifest_id": "kb-2026-08-01"}],
    "tools": [
        {"name": "search_kb", "hitl": False},
        {"name": "create_draft_ticket", "hitl": True},
    ],
    "data_classes": ["customer_tickets", "kb_articles"],
    "owners": {"product": "A. Rivera", "safety": "K. Chen", "security": "J. Patel"},
    "status": "prod",
    "board_decision": "approve_with_conditions",
    "revisit": "2026-11-01",
}

def register_risk(rid: str, threat_class: str, scenario: str, impact: str,
                  likelihood: str, controls: list[str], owner: str, residual: str) -> dict:
    return {
        "risk_id": rid,
        "system_id": INVENTORY["sys_id"],
        "threat_class": threat_class,  # injection | jailbreak | poisoning | privacy | fairness ...
        "scenario": scenario,
        "impact": impact,
        "likelihood": likelihood,
        "controls": controls,
        "owner": owner,
        "residual_risk": residual,
        "status": "open",
        "review_date": date(2026, 11, 1).isoformat(),
    }

RISKS = [
    register_risk(
        "R-12", "injection",
        "Retrieved ticket text treated as instructions; unwanted tool draft.",
        "high", "medium",
        ["wrap_untrusted", "tool_allowlist", "hitl_create_draft_ticket", "logging"],
        INVENTORY["owners"]["security"],
        "medium",
    )
]

def changes_need_full_board(change_type: str) -> bool:
    return change_type in {"new_tool_side_effect", "new_high_risk_use_case", "prod_fine_tune"}

Change Control Thresholds

Expedited (still logged)

Copy edits to user-facing help text
Raising retrieval \(k\) within policy cap
Eval-only dashboard tweaks

Full board

New side-effecting tool or MCP server
Fine-tune / unsigned adapter
Expanding from internal users to public
Training on user content (privacy + copyright)

Related Lectures

Lecture	Role
Responsible AI	Practices the board enforces
Model poisoning	Inventory digests + index manifests
Tool calling	Tools are inventory rows, not afterthoughts
Transparency	What you can honestly tell users
Compliance	Next: regulatory engineering implications

Common Misconception

“Governance is a yearly PDF.” Inventories rot weekly as prompts and indexes change. Second: listing the vendor model name is a complete inventory—missing tools and corpora. Third: the board exists to rubber-stamp launches; a credible board can say no. Fourth: risk registers that never change after launch. Fifth: security-only governance; fairness and privacy risks belong on the same register.

Knowledge Check

Short Answer: What is AI governance here? Answer: Roles, forums, and records that decide which AI systems run, with what controls and owners.
True/False: A foundation-model name alone is a complete inventory. Answer: False—include prompts, indexes, tools, data, owners.
Multiple Choice: A review board’s job includes: (a) go/no-go and residual risk, (b) tuning AdamW, (c) sharding GPUs. Answer: (a).
Short Answer: Name two fields on a risk register row. Answer: Threat class, scenario, impact, likelihood, controls, owner, residual risk (any two).
True/False: HITL reviewers need an operational owner, not only a design checkbox. Answer: True.
Multiple Choice: Adding a side-effecting tool typically needs: (a) full board path, (b) only a temperature change, (c) dropping logs. Answer: (a).
Short Answer: Why pin model digests in the inventory? Answer: Integrity / poisoning control and reproducible production state.
True/False: This lecture replaces legal counsel. Answer: False.
Multiple Choice: Next lecture: (a) compliance, (b) UMAP, (c) beam search. Answer: (a).
Short Answer: Link responsible AI to governance in one sentence. Answer: Governance assigns owners and artifacts that enforce responsible-AI practices.

Key Takeaways

Governance = roles + board + inventory + risk register + change control.
Inventory the system (prompts, RAG, tools), not only the vendor model.
Risks cite Vol. 20 threat classes and named control owners.
Boards can reject or condition launches; records must be auditable.
Next: Compliance.

Trainer’s Guide

Lab: Students fill one inventory row and five risk-register rows for a fictional agent (search + draft ticket + optional refund tool). They mark which changes need full board vs expedited path. Role-play a 15-minute board: approve-with-conditions only.

Whiteboard: RACI across product, safety, security, ops. Draw arrows from each Vol. 20 threat class to a register ID.

Recap: Governance makes responsible AI interruptible and auditable—owners, boards, inventories, and risks. Translate those records into regulatory engineering next in Compliance.