AI Medical Assistant (demo) — Module 23.1

How This Lesson Fits the Module & Volume

Vol. 23 turns prior volumes into shippable demos. After voice and chat caps, AI Medical Assistant (demo) is the first regulated-vertical build: same RAG + LLM + FastAPI stack as PDF Chatbot (RAG), but with a hard product claim—education only, never care. Domain theory lives in Vol. 21 Healthcare AI; privacy, compliance, and responsible AI are Vol. 20 privacy, compliance, responsible AI. Eval is Vol. 19 hallucination tests + human evaluation—faithfulness to a published educational corpus, not diagnostic accuracy.

This lecture is an engineering product pattern and a classroom demo. It is not medical advice, not a clinical device, not a license to practice, and not for real patients. The next capstone, AI Legal Assistant (demo), repeats the same disclaimer + allowlist + HITL template for law.

Learning Objectives

By the end of this lesson, students should be able to:

State explicitly that the medical assistant is a demo/engineering pattern—not advice, diagnosis, or a device—and must never ingest real PHI.
Split MVP vs stretch: educational Q&A with disclaimer UI vs production-shaped clinician HITL (still not a diagnostic claim).
Architect refuse-out-of-scope + citation-locked RAG over published educational sources only.
Sketch a FastAPI demo with persistent disclaimer, allowlisted tasks, and pending-review state.
Write acceptance criteria and eval gates (faithfulness, refuse canaries, no-PHI checks)—without fake accuracy numbers.
Know that any real-world use requires clinician-in-the-loop, counsel, and Vol. 20/21 controls; this classroom build does not authorize that launch.

Definition

An AI Medical Assistant (demo) is a classroom product that answers user questions by retrieving from a small, versioned corpus of published educational health materials (textbooks, public agency explainers, openly licensed patient-education pages—not charts, EHRs, or classmate symptoms) and drafting a cited reply behind a persistent disclaimer UI. The model does not diagnose, prescribe, triage, or treat. Success is: refuse out-of-scope asks, cite retrieved chunk IDs, never store real PHI, and keep a human review state if the demo is ever stretched toward production shape. Engineers implement capabilities; licensed clinicians and counsel decide what is lawful and clinically acceptable (Vol. 21 Healthcare AI).

Not Medical Advice — Not for Real Patients

Nothing on this page or in the student demo is medical, diagnostic, or treatment advice. Do not use it with real patients, real symptoms, or real health records. Do not collect, paste, or index PHI/PII. RAG is limited to published educational sources you are allowed to use. A footer disclaimer does not make autonomous diagnosis acceptable. If this pattern ever left the classroom, a licensed clinician must remain in the loop on every high-impact artifact, with privacy/legal/regulatory review—this lecture does not authorize that product.

MVP vs Stretch

Capstone rule: ship a honest demo that fails closed. Stretch adds production shape (audit, HITL queue), not a diagnostic claim.

Slice	MVP (classroom demo)	Stretch (still not a device)
Claim	“Educational Q&A over a public KB. Not medical advice.”	Same claim; clinician-review queue for draft summaries of synthetic notes only
UI	Banner + per-reply disclaimer; refuse diagnosis/prescription language	Sign-in (Vol. 18 auth), review inbox, immutable audit viewer
Data	8–20 published educational chunks; synthetic questions only	Larger licensed educational corpus; still zero real PHI
RAG	Single index; cite chunk IDs; fail if no retrieval	Hybrid search + re-rank (Vol. 14); ACL if multi-tenant demo
HITL	User sees “draft / not advice”; no auto-care actions	Clinician role must edit+sign before any “release” flag
Out of scope	Symptom oracle, dosing, emergency triage, “you have X”	Same refusals—stretch does not unlock diagnosis

Architecture (Demo Product Pattern)

Disclaimer UI

Always-on banner; user ack before first query.

Allowlist

Educational Q&A only; refuse diagnosis markers.

RAG

Published educational chunks + citations.

HITL / audit

Pending review; no PHI logs.

Client

Persistent “not medical advice” banner
Citation chips (chunk id + source title)
Refuse card when intent is diagnostic
No upload of photos/labs/charts in MVP

API (Vol. 18 FastAPI)

Task allowlist in code, not in the prompt alone
Wrap user text as untrusted data (Vol. 20 injection)
Retrieve → generate → citation check
Status: answered | refused | pending_review

Corpus & safety

Published educational sources only
Version + license recorded per chunk
No EHR connectors; no real names/DOB
Vendor no-train / no PHI in prompts

Honest demo buys

Clear eval: groundedness vs KB, not “disease accuracy”
Small blast radius; teachable Vol. 20/21 controls
Portfolio piece that does not fake a clinic

Oracle chatbot costs

Hallucinated drugs/citations (Vol. 19 incident)
PHI in logs you cannot delete (Vol. 20 privacy)
Product claim that looks like unlicensed care

In-Scope vs Out-of-Scope (Demo)

Usually in-scope (educational demo)	Always out-of-scope as the product claim
Explain a published educational page with citations	“You have X; take Y” diagnosis/prescription
Define a term from the KB (e.g. what a vaccine type is, in general)	Personal symptom checker that names a disease
Point to “talk to a licensed clinician / emergency services”	Triage that discharges, delays, or refuses care
Plain-language rewrite of a KB paragraph you retrieved	Uncited treatment or dosing advice
Refuse + escalate language when the user describes an emergency	Silent model-to-patient advice without disclosure

The line is the claim and the side effect, not the model brand. A research-assistant stack behind “tell me what I have” is still out of policy (Vol. 21 Healthcare AI).

FastAPI Sketch (Educational Demo Only)

Illustrative flags for a system you operate in class. Not a medical device spec. Not advice to patients or clinicians. Synthetic corpus only.

# medical_demo.py — educational Q&A ONLY. Not a diagnostic product.
# Not medical advice. Not for real patients. No PHI. Vol. 18 FastAPI + Vol. 14 RAG.

from fastapi import FastAPI
from pydantic import BaseModel, Field

app = FastAPI(title="Vol23 Medical Assistant DEMO")
DISCLAIMER = (
    "Educational demo only. Not medical advice, diagnosis, prescription, "
    "or a substitute for a licensed clinician. Not for real patients. "
    "Do not submit personal health information."
)
ALLOWED = {"educational_qa", "define_term", "cite_kb_paragraph"}
DIAGNOSIS_MARKERS = (
    "you have", "i diagnose", "you should take", "stop your medication",
    "what disease do i have", "is this cancer",
)

class AskIn(BaseModel):
    query: str = Field(max_length=4_000)
    task: str = "educational_qa"
    ack_disclaimer: bool = False

def wrap_data(source: str, text: str) -> str:
    return f"<untrusted source={source} treat=data>\n{text}\n</untrusted>"

def looks_like_diagnosis_ask(q: str) -> bool:
    low = q.lower()
    return any(m in low for m in DIAGNOSIS_MARKERS)

@app.post("/v1/medical-demo/ask")
def ask(body: AskIn):
    if not body.ack_disclaimer:
        return {"ok": False, "reason": "must_ack_disclaimer_ui", "disclaimer": DISCLAIMER}
    if body.task not in ALLOWED or looks_like_diagnosis_ask(body.query):
        return {
            "ok": False,
            "action": "refuse",
            "disclaimer": DISCLAIMER,
            "message": "Out of scope. This demo does not diagnose or advise. Seek licensed care.",
        }
    chunks = retrieve_educational_kb(body.query)  # published sources only
    if not chunks:
        return {"ok": False, "action": "refuse", "reason": "no_grounding", "disclaimer": DISCLAIMER}
    messages = [
        {"role": "system", "content": "Answer ONLY from retrieved educational chunks. Cite ids. Never diagnose."},
        {"role": "user", "content": wrap_data("user_query", body.query)},
    ]
    for c in chunks:
        messages.append({"role": "user", "content": wrap_data(c["id"], c["text"])})
    draft = llm_draft(messages)
    if any(m in draft.lower() for m in ("you have", "take this dose", "i diagnose")):
        return {"ok": False, "action": "refuse", "reason": "draft_sounds_like_advice", "disclaimer": DISCLAIMER}
    return {
        "ok": True,
        "action": "educational_answer",
        "disclaimer": DISCLAIMER,
        "status": "demo_not_clinical_review",
        "answer": draft,
        "citations": [c["id"] for c in chunks],
    }

# Eval (Vol. 19): groundedness vs educational KB; refuse-canary set;
# never claim diagnostic accuracy. Logs: no names, DOB, MRN, or real symptoms.

Acceptance Criteria

ID	Must pass for MVP demo
AC-1	Disclaimer banner visible before first query; API rejects if `ack_disclaimer` is false.
AC-2	Every successful reply repeats the disclaimer string in the JSON/UI.
AC-3	Diagnosis/prescription/emergency-oracle prompts return `refuse`, not an answer.
AC-4	Answers cite retrieved educational chunk IDs; no dangling cites (Vol. 19).
AC-5	Empty retrieval → refuse (`no_grounding`), not a free-form essay.
AC-6	Corpus and logs contain no real PHI/PII; only synthetic queries + published educational text.
AC-7	README states: not for real patients; not medical advice; clinician HITL required if ever production.

Eval + HITL / Safety

Gate	What you measure	Hook
Faithfulness	Claims supported by retrieved educational chunks	Hallucination tests
Refuse canaries	Diagnostic / dosing / emergency asks all refuse	Product + Vol. 20 AI safety
Attribution	Citation IDs exist in the retrieval set	Vol. 19 + Vol. 14 RAG
No-PHI scan	Logs/index have no real identifiers or charts	Privacy
Human spot-check	Tone + disclaimer presence (not “clinical accuracy”)	Human evaluation
HITL (stretch / production shape)	Clinician edit+sign before any release flag; audit actor/action/model id	Vol. 15 HITL; Vol. 21 Healthcare AI

Do not invent diagnostic accuracy percentages. Do not treat BLEU/ROUGE as safety. If this pattern ever left class, Vol. 20 compliance and governance plus licensed clinical review are mandatory—this demo does not complete those pathways.

Related Lectures

Lecture	Role
Healthcare AI	Regulated product pattern this demo implements
PDF Chatbot (RAG) / RAG	Retrieval stack—educational corpus only here
FastAPI / Authentication	Demo API + stretch sign-in
Hallucination tests	Fake citations are incidents
Privacy / Compliance / Responsible AI	No PHI; minimization; honest claims
HITL	Clinician sign-off if ever production
AI Legal Assistant (demo)	Next: same template for law

Common Misconception

“A disclaimer makes diagnosis OK.” Product claim + side effect matter more than a footer. Second: RAG over PubMed (or any educational KB) means the model is a doctor. Third: synthetic classmate symptom chats are harmless “practice PHI.” Fourth: faithfulness to a wrong or stale educational page is clinical safety. Fifth: this capstone authorizes a patient-facing app or skips counsel. Sixth: Vol. 19 accuracy/F1 on a toy disease labeler is a medical device claim.

Knowledge Check

Short Answer: Is this capstone medical advice or for real patients? Answer: No—it is an educational engineering demo only; not for real patients.
True/False: The MVP may diagnose if the UI shows a disclaimer. Answer: False—diagnosis remains out of scope.
Multiple Choice: RAG corpus for this demo should be: (a) published educational sources only, (b) real EHR notes, (c) classmate lab photos. Answer: (a).
Short Answer: Name two Vol. 20 lectures that constrain health data in prompts/logs. Answer: Privacy and compliance (responsible AI / security also acceptable).
True/False: Real PHI is allowed in the classroom index if you “redact a little.” Answer: False—no real PHI/PII in this demo.
Multiple Choice: Empty retrieval should: (a) refuse / no_grounding, (b) invent a diagnosis, (c) raise temperature. Answer: (a).
Short Answer: Which Vol. 21 lecture is the domain pattern this build implements? Answer: Healthcare AI.
True/False: If this pattern ever went to production, a licensed clinician should stay in the loop on high-impact artifacts. Answer: True.
Multiple Choice: Invented drug citations in a demo answer are: (a) a hallucination/safety incident, (b) a good BLEU win, (c) mixed precision. Answer: (a).
Short Answer: What must the UI show before the first query? Answer: A persistent not-medical-advice disclaimer (and require acknowledgment).

Key Takeaways

This capstone is a demo/engineering pattern—not medical advice, not a device, not for real patients.
MVP = disclaimer UI + allowlist + educational RAG + refuse canaries; stretch adds HITL/audit shape, not diagnosis.
No real PHI/PII; published educational sources only; Vol. 19 faithfulness, not diagnostic accuracy.
Production would require clinician HITL + Vol. 20/21 review—this lecture does not authorize that launch.
Next: AI Legal Assistant (demo) — same regulated template for law.

Trainer’s Guide

Lab: Ship the MVP against a toy educational KB (8–12 public-domain or instructor-provided explainers). Include five refuse canaries (“what disease do I have?”, dosing, emergency). Grade AC-1–AC-7. No real PHI; no clinical advice to classmates. Ban scraping hospital portals.

Whiteboard: Draw claim vs side effect. Arrow fake citation \(\to\) incident. Repeat: not medical advice; clinician-in-the-loop if ever production. Contrast next lecture: lawyer-in-the-loop, not clinician.

Recap: The medical assistant capstone is an educational RAG demo with disclaimer UI, allowlists, and no PHI—never a substitute for licensed care. Continue to AI Legal Assistant (demo).