The clone chats; the PDF chatbot grounds in files. This capstone is structured generation + human approval: a JSON resume as source of truth, an LLM that rewrites (tone, bullets, JD alignment), and an export that must not fire until a human clicks approve. It is Vol. 21 Document AI + Vol. 13 structured output / JSON prompting, with Vol. 15 HITL on the only irreversible step—download / apply.
ATS (applicant tracking systems) are not one algorithm. This lecture teaches caveats, not fake “98% ATS scores.” Vol. 20 privacy (CVs are PII) and fairness / bias apply: do not invent employment, degrees, or metrics. Next: code assistant (diffs + tests as judge).
Learning Objectives
By the end of this lesson, students should be able to:
- Model a resume as versioned JSON (schema), not as a free-form blob the model owns.
- Separate extract / rewrite / export stages with validation in code.
- Require human approve-before-export; never auto-submit to a job portal.
- State ATS caveats without inventing vendor scores or guaranteed pass rates.
- Eval factuality (no invented jobs) separately from style/JD keyword coverage.
- Apply Vol. 20 PII + bias controls to a career document product.
An AI resume builder is a document product that stores the candidate’s facts in a structured JSON resume, proposes LLM rewrites (summary, bullets, skills grouping) as diffs against that JSON, and exports PDF/DOCX/Markdown only after human approval. It is not an ATS oracle, not a background-check, and not licensed career counseling. Hallucinated employment is a safety failure, not a creative feature.
Problem and Scope
Job seekers need tailored language for a posting without lying. Recruiters and ATS parsers want consistent headings and plain text. The conflict: models love impressive verbs and invented impact numbers. The product job is constrained rewrite plus an audit trail of what the human accepted.
| MVP (done when…) | Stretch | |
|---|---|---|
| Source of truth | JSON Resume-shaped schema (basics, work[], education[], skills[]) | Projects, publications, i18n locales |
| Ingest | Form editor + optional paste/PDF extract → JSON (HITL confirm extract) | LinkedIn export parse; OCR scans |
| Rewrite | JD paste → proposed bullet/summary diff; user accept/reject per field | Cover letter draft with same HITL; multi-JD variants |
| Export | PDF/Markdown from approved JSON only | DOCX template; shareable link with expiry |
| Out of scope | No auto-apply; no fake ATS %; no fabricating metrics | Job-board submit APIs (still HITL if ever added) |
JSON is canonical
- Validate with Pydantic / JSON Schema
- LLM output must parse or retry
- PDF is a projection, not the store
ATS caveats (honest)
- No single “the ATS” scoring model
- Prefer simple headings, selectable text, no text-in-images
- Keyword stuffing ≠ relevance; humans still read
- Do not sell a numeric ATS score
Vol. 22 / 18 stack
- FastAPI + same auth as clone
- OpenAI-compatible JSON mode or HF + schema repair
- Postgres JSONB or SQLite
- WeasyPrint / reportlab / md→PDF for export
Schema + HITL (win)
- Factuality checkable field-by-field
- Export is deterministic from JSON
- Audit: who approved which rewrite
One-shot pretty PDF (lose)
- Invented dates hide in prose
- Cannot diff versions
- ATS + humans both suffer from layout gimmicks
Architecture
Form / extract → validated JSON v0.
JD + JSON → proposed patch.
Accept / edit / reject per field.
Render only status=approved.
| Plane | Responsibility |
|---|---|
| UI | Split view: JSON/form left, JD + diff right, Approve export disabled until review |
| API | /v1/resumes, /v1/resumes/{id}/rewrite, /v1/resumes/{id}/approve, /v1/resumes/{id}/export |
| Model | JSON-constrained chat; temperature low; no tools that hit job boards |
| Storage | Resume versions + patch proposals + approval events (who, when) |
| Eval | Factuality, schema validity, bias/tone sample, export fidelity |
Concrete Stack + Implementation Sketch
Reuse FastAPI auth/quotas. Store JSONB. Use structured outputs (OpenAI-compatible response_format or tool-call schema). Export is a pure function of approved JSON—no second LLM pass that can re-hallucinate dates.
Acceptance Criteria (“Done When…”)
| # | Done when… |
|---|---|
| 1 | User can create/edit JSON resume via form; invalid schema is rejected in API, not silently stored. |
| 2 | Rewrite against a JD returns a proposal; export endpoint returns 409 until approve. |
| 3 | A rewrite that inserts a new employer or degree is rejected by fact_guard (422), even if the model emitted it. |
| 4 | Approve writes an audit row (user, timestamp, proposal id); UI diff is reviewable field-by-field. |
| 5 | Exported PDF text is selectable (not a screenshot); headings are conventional (Experience, Education, Skills). |
| 6 | Product copy has no numeric ATS score; a short “ATS caveats” note is visible near export. |
| 7 | PII: resumes are owner-scoped; delete works; no CV text in vendor logs beyond stated retention. |
Eval Rubric + HITL / Safety
| Gate | What you measure | Hook |
|---|---|---|
| Factuality | No new employers/degrees/metrics vs source JSON | Code guard + Vol. 19 hallucination tests |
| Schema validity | 100% parse rate after one retry | Vol. 13 structured output |
| JD alignment (human) | Bullets relevant without stuffing; 5 raters on a rubric | Human eval |
| Bias / tone | No gendered self-sabotage prompts; no “fix your ethnicity” features | Vol. 20 bias / fairness |
| HITL completeness | Export impossible without approve; no job-board side effects | Vol. 15 HITL |
| Privacy | Owner isolation, TTL/export/delete, minimize JD+CV in logs | Vol. 20 privacy |
HITL is the product: the human is the author of record. The model is a copy editor with a fact fence. Auto-applying to Greenhouse/Lever is out of scope; if a later team adds it, that is an irreversible tool and needs a second confirmation.
Related Lectures
| Lecture | Role |
|---|---|
| JSON prompting / structured output | Schema discipline |
| Document AI / HITL | Extract + approve pattern |
| FastAPI / Docker | Serving |
| Privacy / bias | CVs are sensitive |
| PDF chatbot / Code assistant / Interview assistant | Siblings |
“If the PDF looks premium, ATS will rank it #1.” Many parsers want simple text; we do not invent scores. Second: the model may add a plausible internship “to help the student.” That is a factuality fail. Third: export can run on pending_review to “save a click.” Fourth: keyword stuffing is the same as JD alignment. Fifth: cover letters can auto-send. Sixth: JSON is optional if the markdown looks good.
Knowledge Check
- Short Answer: What is the source of truth in this product? Answer: Versioned structured JSON resume, not the PDF blob.
- True/False: Export may run on a pending rewrite to speed up users. Answer: False—409 until human approve.
- Multiple Choice: Inventing a new employer in a rewrite should be: (a) rejected in code (fact_guard), (b) allowed if the JD requires it, (c) scored as +ATS points. Answer: (a).
- Short Answer: Name one honest ATS caveat. Answer: No single ATS score; prefer selectable text/simple headings; keyword stuffing ≠ relevance; humans still read (any valid).
- True/False: This lecture publishes a fake 98% ATS pass rate. Answer: False.
- Multiple Choice: PDF rendering should: (a) be a pure function of approved JSON, (b) call the LLM again to “polish dates,” (c) screenshot a canvas. Answer: (a).
- Short Answer: Why are CVs a Vol. 20 privacy topic? Answer: They contain PII (identity, employment, often phone/address); owner-scope, delete, minimize logs.
- True/False: Education fields may be freely rewritten including new degrees. Answer: False—education mutation is blocked in the sketch.
- Multiple Choice: Next sibling capstone is: (a) AI Code Assistant, (b) Midjourney, (c) n8n. Answer: (a).
- Short Answer: Where does HITL sit in this architecture? Answer: On approve-before-export (and on confirming extracted facts); no unsupervised job-board submit.
Key Takeaways
- Resume AI is schema + constrained rewrite + human approve, not a magic PDF.
- Fact fences live in code; the model cannot become the employment oracle.
- ATS advice is qualitative; never invent scores or guaranteed ranks.
- Export is a projection of approved JSON; audit who accepted which patch.
- Next: repo diffs with tests as judge—AI Code Assistant.
Lab: Students use a fictional resume (no real SSN/phone). Implement schema, rewrite, fact_guard, approve, Markdown or PDF export. Red-team: prompt the model to add “Google intern, Summer 2024” and show the 422. Deliverable: eval of 8 rewrites (factuality Y/N, schema Y/N, human style 1–5) + ATS caveats paragraph with zero invented percentages.
Exit ticket: “Marketing wants a circular gauge that says ATS Match 94%. What do you ship instead?”
Recap: The resume builder capstone stores facts in JSON, rewrites under a fact fence, and exports only after HITL—with honest ATS caveats. Continue to AI Code Assistant.