Deployment — Module 18.2

How This Lesson Fits the Module & Volume

You have an API, FastAPI (or Flask), image, optional K8s, Redis/Celery, streaming, and auth. Deployment is the repeatable path from git to a named environment (dev/staging/prod) with the right secrets, URLs, and rollback. It is not “it ran on my laptop” and not yet full observability.

Vol. 17’s A1111 on port 7860 was a studio. This lecture is how the wrapper around Module 18.1 SDKs actually ships. Next two lectures: monitoring then observability—so you know the deploy worked.

Learning Objectives

By the end of this lesson, students should be able to:

Describe a CI → image → registry → env promote pipeline.
Separate config (model SKU, log level) from secrets (vendor keys, JWT).
Choose VM+Compose vs PaaS vs Kubernetes for a given stage.
Run health checks and rolling/blue-green style cutovers without dropping auth.
Keep .env out of git; inject at runtime.
Plan GPU infer deploys as a different workload than the chat proxy.

Definition

Deployment is releasing a specific artifact (container tag, sometimes a serverless bundle) into an environment with bound configuration and secrets, plus a defined rollback. An environment is an isolated slice (dev/staging/prod) with its own URLs, keys, and data. Deploying is not training, not prompt tuning, and not “we docker build on the prod SSH box by hand” if you can avoid it.

Artifact Pipeline

Step	Output	Notes
CI test + lint	green build	Fail closed before an image exists
Docker build/push	`chat-api:gitsha`	Pin digest; never only `latest`
Deploy staging	live staging URL	Staging vendor keys / budget caps
Smoke	`/health` + one auth’d chat	Prove JWT + upstream LLM
Promote prod	same image digest	Config/secrets change, not code
Rollback	previous digest	K8s rollout undo / PaaS revert

Env Files vs Runtime Inject

# .env.example  (COMMIT THIS)
OPENAI_API_KEY=
ANTHROPIC_API_KEY=
GEMINI_API_KEY=
JWT_SECRET=
REDIS_URL=redis://redis:6379/0
DEFAULT_MODEL=gpt-4.1-mini
LOG_LEVEL=info

# .env  (DO NOT COMMIT) — local only
# Production: cloud secret manager / K8s Secret / PaaS env UI

# GitHub Actions sketch (teaching)
# - docker build -t ghcr.io/org/chat-api:${{ github.sha }} .
# - docker push ...
# - kubectl set image deploy/chat-api api=ghcr.io/org/chat-api:${{ github.sha }}
# - kubectl rollout status deploy/chat-api

# PaaS alternative (Render/Fly/Cloud Run):
#   same image, env vars in dashboard, min instances for cold-start LLMs
# VM + Compose:
#   docker compose --env-file /etc/chat-api/prod.env up -d
#   caddy/nginx TLS in front; still not a substitute for auth

# Smoke after deploy:
# curl -sf https://api.example/health
# curl -sf -H "Authorization: Bearer $STAGING_JWT" \
#   -H "Content-Type: application/json" \
#   -d '{"prompt":"ping"}' https://api.example/v1/chat

Where to Run It

PaaS / Cloud Run

Fastest for SDK-proxy APIs
Watch cold starts + timeouts
Great class → startup path

K8s

API + Redis + Celery together
GPU pools later (18.3)
Needs rollout discipline

Single VM

Compose + Caddy
Fine until HA matters
Still use tagged images

Good deploys

Immutable tags, smoke tests, rollback
Secrets injected; configs per env
Staging spends cheap model SKUs

Bad deploys

git pull + python main.py on prod
Same OpenAI key for student laptops and prod
No health check, no owner, no budget cap

Related Lectures

Lecture	Role
Docker / K8s	Artifact + scheduler
Authentication	JWT_SECRET rotation on deploy
Monitoring	Know if the new tag is healthy
GPU	Separate infer deploy, not the proxy

Common Misconception

“Deployed means Docker Compose up on a public IP.” Without TLS, auth, secrets hygiene, and a rollback, that is a demo. Second: rebuilding images on the prod VM with host .env copied into layers. Third: promoting by rebuilding (new digest) instead of promoting the staging digest. Fourth: deploying A1111 and calling it the product API.

Knowledge Check

Short Answer: What artifact should staging and prod share when promoting? Answer: The same image digest/tag (config/secrets may differ).
True/False: Commit real .env with production keys. Answer: False—commit .env.example only.
Multiple Choice: A minimal smoke test includes: (a) /health + one auth’d chat, (b) only docker ps, (c) t-SNE. Answer: (a).
Short Answer: Why cheap model SKUs on staging? Answer: CI/smoke traffic should not burn prod-rate tokens.
True/False: latest is a sufficient production pin. Answer: False—use git SHA/digest.
Multiple Choice: GPU infer should usually be: (a) a separate workload, (b) inside nginx, (c) a JWT claim. Answer: (a).
Short Answer: Name one rollback mechanism. Answer: Any of: kubectl rollout undo, PaaS revert, redeploy previous digest.
True/False: PaaS is invalid for an OpenAI-proxy API. Answer: False—often the simplest good path.
Multiple Choice: Next lecture: (a) Monitoring, (b) DALL·E, (c) Lasso. Answer: (a).
Short Answer: Config vs secret example? Answer: DEFAULT_MODEL / LOG_LEVEL vs OPENAI_API_KEY / JWT_SECRET.

Key Takeaways

Deploy = tagged artifact + env-specific config/secrets + smoke + rollback.
Promote digests; do not rebuild prod by hand.
PaaS, VM+Compose, or K8s—choose by ops need, not fashion.
Chat proxy ≠ GPU infer deploy.
Next: Monitoring.

Trainer’s Guide

Lab: Push an image to a registry (or local kind load). Deploy staging, smoke with JWT, change DEFAULT_MODEL via config only, redeploy, rollback.

Whiteboard: Git SHA → image → staging → prod. Two secret clouds. Cross out “SSH and vim main.py.”

Recap: Deployment promotes immutable images with injected secrets. Prove it works with Monitoring.