The travel planner gated irreversible bookings. This capstone gates judgments about people. Vol. 21 Education AI split tutor vs assessment; Vol. 20 bias, fairness, and responsible AI apply immediately. MVP = practice interviewer (candidate rehearses against a rubric). Stretch may add interviewer-aid notes for a human interviewer. Neither mode is a hiring oracle.
The curriculum closer is next: document analyzer—multi-doc ingest, schema, diff, redaction, and a full-stack recap of Volumes 11–22.
Learning Objectives
By the end of this lesson, students should be able to:
- State the MVP explicitly: practice interviewer, not automated hiring.
- Author a visible scoring rubric (dimensions + scale + evidence quotes).
- Show a bias / fairness warning in the UI; refuse protected-class inferences.
- Label every score “practice feedback, not a hiring decision.”
- Eval rubric consistency on gold answers—not “predict who gets the job.”
- Describe interviewer-aid as stretch-only, still HITL, still not an oracle.
An AI interview assistant in this lecture’s MVP is a practice interviewer: given a job description and a declared rubric, it asks questions, accepts answers, and returns dimension scores plus evidence spans and coaching notes. It is not a hiring oracle—it must not rank candidates for an offer, infer protected attributes, or auto-reject. Interviewer-aid (stretch) helps a human interviewer take structured notes against the same rubric; the human still owns the hiring decision (Vol. 15 HITL + Vol. 20 fairness).
Build candidate practice first. Do not ship a silent scoring API that a recruiter could paste into an ATS as a decision. If you add interviewer-aid later, keep the disclaimer, log that a human must confirm, and never emit a single “hire/no-hire” token as the product output.
Problem, MVP, and Stretch
| MVP — practice interviewer | Stretch — interviewer-aid | |
|---|---|---|
| User | Candidate rehearsing | Human interviewer + candidate (live notes) |
| Inputs | JD + rubric + answer text (or STT optional) | Same + interviewer timestamps |
| Outputs | Questions, per-dimension scores, evidence, coaching | Note draft + rubric assist for the human |
| Decision | None. Explicit non-hiring label | Human-only hire/no-hire; AI never writes the offer |
| Bias controls | Warning UI; no demographic questions; no accent/name scoring | Same + Vol. 20 fairness review sample |
| Out of scope | ATS auto-reject; personality “psychometrics”; deepfake avatars as truth | Surveillance, emotion recognition as hire signal |
Practice interviewer (MVP)
- Helps the learner improve answers
- Rubric is a coaching contract
- Scores are feedback, not offers
- Safe classroom / portfolio default
Interviewer-aid (stretch)
- Helps the interviewer stay consistent
- Notes + evidence for human review
- Still not an oracle
- Higher Vol. 20 scrutiny
Hiring oracle (forbidden)
- Single hire/no-hire score
- Hidden rubric
- Demographic or proxy features
- Auto-reject without a human
Visible rubric buys
- Evalable dimensions (STAR, correctness, clarity)
- Candidate can contest a score with evidence
- Easier bias review than a black-box “fit” number
Holistic vibe score costs
- Unstable across paraphrases
- Easy to smuggle accent/name bias
- Looks like a hiring decision in a dashboard
Architecture
| Layer | MVP choice | Notes |
|---|---|---|
| UI | JD paste, rubric editor, chat questions, score cards, disclaimer banner | Banner always visible on score views |
| API | FastAPI: /session/start, /turn, /score | mode=practice required |
| Rubric | JSON: dimensions, 1–5 scale, descriptors | Version the rubric like an API |
| Model | Chat completion + structured scores | Vol. 13 JSON; Vol. 22 vendor pick |
| Optional STT | Off in MVP; text answers | Vol. 16 if added; do not score accent |
| Storage | Session transcript + rubric_id + scores + disclaimer_ack | No protected-class fields |
| Eval | Agreement vs gold rubric on fixtures; banned-inference tests | Not “who would be hired” |
FastAPI Sketch: Rubric Scoring (Practice Mode)
Acceptance Criteria (“Done When…”)
| # | Criterion | How you prove it |
|---|---|---|
| 1 | Mode is practice | API/UI says practice interviewer; no hire/no-hire enum |
| 2 | Rubric visible | Every score maps to a named dimension + scale |
| 3 | Disclaimer ack | Start without ack → 403; disclaimer on every score payload |
| 4 | Evidence from the answer | Evidence string is a substring of the answer (normalized) |
| 5 | No protected-class scoring | Fixture with demographic bait → 422 or ignore + warn, never a dim score on identity |
| 6 | hiring_decision is always null | Schema + test |
| 7 | Gold consistency | Two gold answers: strong vs weak differ in the expected direction on at least one dim |
Eval, HITL, and Safety
Eval rubric reliability (same answer scored twice should not wildly flip) and sensitivity (weak vs strong gold). Do not eval “accuracy vs who was hired historically”—that encodes past bias (Vol. 20). Human evaluation on coaching usefulness. Interviewer-aid stretch: human must confirm notes before they enter a hiring file.
| Risk | Control |
|---|---|
| Treated as a hiring oracle | Disclaimer, null hiring_decision, practice mode only in MVP |
| Bias / proxy discrimination | No demographic fields; refuse identity bait; Vol. 20 fairness sample |
| Accent / voice as competence | Text MVP; if STT added, do not score fluency-as-intelligence |
| Hidden rubric | Dimensions always returned to the user |
| Prompt injection in JD | Wrap JD + answer as untrusted data |
Related Lectures
| Lecture | Role |
|---|---|
| Education AI | Tutor vs assessment integrity analog |
| Bias / fairness / responsible AI | Non-oracle + non-discrimination |
| Structured output | Rubric JSON |
| HITL | Human owns real hiring |
| Human evaluation | Coaching quality |
| Travel planner / Document analyzer | Prev / curriculum closer |
“A 1–5 score is objective if the model is large.” Rubrics still encode values; models still drift. Second: matching historical hire labels is a good eval (it often replays bias). Third: asking the model for race/gender “only to debias” in the MVP. Fourth: interviewer-aid can auto-reject to save time. Fifth: scoring accent or “confidence” from audio is a harmless extra. Sixth: hiding the rubric from the candidate makes scores more valid—it mostly makes them less contestable.
Knowledge Check
- Short Answer: What is the explicit MVP mode for this lecture? Answer: Practice interviewer (candidate rehearsal), not hiring automation.
- True/False: The MVP may return a hire/no-hire decision. Answer: False—
hiring_decisionstays null. - Multiple Choice: Interviewer-aid belongs in: (a) stretch with HITL, (b) MVP auto-reject, (c) CUDA kernels. Answer: (a).
- Short Answer: Name two Vol. 20 topics that constrain this product. Answer: Any two of: bias, fairness, responsible AI, privacy, transparency.
- True/False: Historical “who was hired” accuracy is the recommended eval. Answer: False—it can encode past bias.
- Multiple Choice: Scores without a visible rubric are: (a) a product failure for this capstone, (b) more scientific, (c) required by softmax. Answer: (a).
- Short Answer: What must every score payload include besides numbers? Answer: Disclaimer + dimension evidence/coaching (and no hiring decision).
- True/False: Protected-class inference is in scope if it improves “culture fit.” Answer: False.
- Multiple Choice: Closest Vol. 21 analog for tutor vs high-stakes judgment: (a) Education AI, (b) image generators, (c) n8n. Answer: (a).
- Short Answer: Which lecture closes the entire curriculum after this one? Answer: AI Document Analyzer.
Key Takeaways
- MVP is a practice interviewer with a visible rubric—not a hiring oracle.
- Disclaimer, null hiring_decision, and banned demographic scoring are acceptance tests.
- Eval rubric consistency on gold answers; do not optimize against historical hire labels.
- Interviewer-aid is stretch + HITL only; humans still own offers.
- Final lecture: AI Document Analyzer (curriculum capstone).
Lab: Provide a toy JD (backend intern) and a 4-dimension rubric (correctness, structure, communication, tradeoff reasoning). Two gold answers: strong STAR vs vague. Students must show disclaimer, null hiring field, and directional score difference. Add a demographic-bait answer that must 422.
Discussion: Why “culture fit” is a bias magnet. If a student wants interviewer-aid, require a written HITL policy before any code.
Recap: The interview assistant MVP is practice-only rubric coaching with bias warnings—not automated hiring. Close the course with AI Document Analyzer.