← Master Index
Vol. 18 Module 18.4 Lecture

Tier 1 — API-Only (no GPU, any laptop, internet + API key)

System Requirements — Basic to Advanced (added)

How This Lesson Fits the Module & Volume

Module 18.3 ended at Edge AI—on-device models. Module 18.4 is a hardware-sizing playbook. Tier 1 is the honest default for most students and many products: no GPU, any laptop, internet + API key. Volume 18.1 SDKs (OpenAI, Anthropic, Gemini) are the runtime. Volume 06 VRAM math and Volume 12 KV/FlashAttention still matter intellectually—you just pay someone else’s cluster to host them.

Stay here until token spend, privacy, latency RTT, or offline requirements force Tier 2+. Buying a 4090 to “learn GenAI” is usually a worse classroom than this tier.

Learning Objectives

By the end of this lesson, students should be able to:

  • State Tier 1 minimums: laptop/CPU, 8+ GB system RAM, browser or Python, internet, billed API key.
  • List what you can ship: chat apps, RAG against a hosted embed+LLM, agents, eval harnesses.
  • List what you cannot: local 7B+ weights, LoRA train, TensorRT, CUDA kernels.
  • Compare API TCO vs idle silicon using a simple monthly token budget.
  • Apply privacy/compliance gates that force leaving Tier 1.
  • Hand off cleanly to Tier 2 when a quantized 7B on a 3060 is actually justified.
Definition

Tier 1 (API-only) is a development and production posture where all model weights live at a provider. Your machine runs the application (Python, FastAPI, a notebook, a browser), stores keys securely, and exchanges tokens over HTTPS. There is no local CUDA, no VRAM budget, and no KV cache on your laptop. The hardware requirement is: a general-purpose computer, network, and the operational discipline of Module 18.2 (auth, logging, rate limits)—not an RTX card.

Bill of Materials (what “any laptop” means)

ItemMinimumNiceNot required
CPU / OSAny x86 or Apple Silicon; Win/macOS/Linux16 GB RAM for browser + IDE + DockerNVIDIA GPU, CUDA, TensorRT
Disk~20 GB for tools + gitSSD; datasets stay in cloud object storeMulti-TB checkpoint shelves
NetworkStable broadband; TLSLow RTT to provider regionInfiniBand, 100 GbE
SecretsAPI key in env / secret managerPer-env keys, spend capsGPU driver matrix
SoftwarePython 3.11+, SDK, HTTP clientDocker (18.2), pytest, eval scriptstorch.cuda, bitsandbytes

What You Can Run vs What You Pretend

In scope

  • Chat, tools, structured output via 18.1 SDKs
  • RAG: hosted embeddings + vector DB SaaS or tiny local index
  • Evals, prompt iteration, product UX
  • CPU-only ONNX micro-models (optional)

Out of scope

  • Local 7B+ generate() (that is Tier 2)
  • QLoRA / full FT (Tiers 4–5)
  • TensorRT engine builds
  • Offline air-gap inference

Leave Tier 1 when

  • Data cannot leave the building
  • Token spend > GPU rent + ops
  • Hard latency < WAN RTT
  • You must customize weights

Minimal Client (no CUDA on purpose)

This is the entire “hardware stack”: HTTP + a key. Compare with Module 18.3’s torch.cuda inventory—if this script is enough for the product, do not buy a GPU.

import os from openai import OpenAI # or anthropic / google-genai — Vol. 18.1 client = OpenAI(api_key=os.environ["OPENAI_API_KEY"]) # never commit keys resp = client.chat.completions.create( model="gpt-4.1-mini", # pick current cheap+good; pin in config messages=[ {"role": "system", "content": "You are a concise curriculum assistant."}, {"role": "user", "content": "When should I leave API-only for a local 7B?"}, ], max_tokens=256, ) print(resp.choices[0].message.content) print("usage", resp.usage) # prompt + completion tokens → monthly TCO # TCO sketch: monthly_usd ≈ (in_tok * in_price + out_tok * out_price) # If monthly_usd consistently > cloud A10/A100 rent + your ops hours, consider Tier 2–5. # Privacy gate: if prompts contain regulated PII that the DPA forbids, stop — do not "just use the API."

TCO and Honesty

Stay API-only

  • Classrooms, MVPs, bursty traffic, frontier quality
  • Zero driver/CUDA/TensorRT on-call
  • You still learn Vol. 06/12 conceptually
  • Spend caps + observability (18.2) are the real ops

Do not stay out of stubbornness

  • Air-gapped hospital/factory weights
  • Steady high QPS where a 3060 Q4 7B is enough quality
  • Need LoRA on private corpora (Tier 4)
  • Sub-50 ms on-LAN latency SLOs

Related Lectures

LectureWhy it sits beside Tier 1
18.3 Edge AIPrevious: on-device ≠ API client
18.1 OpenAI SDK (+ Anthropic / Gemini)The actual runtime
18.2 Authentication / ObservabilityKeys, traces, spend
Vol. 06 VRAMWhat you are not buying yet
Vol. 12 KV CacheProvider still pays this; you pay tokens
Tier 2First honest local GPU rung
Common Misconception

“I cannot learn real AI without a GPU.” Tier 1 is how most production apps start and many stay. Second: “API-only means no architecture knowledge.” You still design RAG, tools, evals, and you must understand why providers bill prompt vs completion (KV/prefill vs decode—Vol. 12). Third: “A MacBook with MPS is still Tier 1.” If you load local weights, you left Tier 1 even without NVIDIA—call it local-CPU/MPS and size it separately; this lecture’s contract is no local LLM weights.

Knowledge Check

  1. Short Answer: What three things does Tier 1 actually require? Answer: A general computer, internet, and a provider API key (plus basic app/runtime).
  2. True/False: Tier 1 requires CUDA and 12 GB VRAM. Answer: False—no local GPU is required.
  3. Multiple Choice: In scope for Tier 1: (a) QLoRA on Mistral-7B, (b) chat + RAG via hosted APIs, (c) TensorRT-LLM engine build. Answer: (b).
  4. Short Answer: Name two reasons to leave Tier 1. Answer: Privacy/air-gap, token TCO vs GPU, latency RTT, or need to train/adapt weights (any two).
  5. True/False: Volume 12 KV cache still exists when you call an API. Answer: True—it lives on the provider GPU; you pay in tokens/latency.
  6. Multiple Choice: Best classroom default: (a) force every student to buy a 4090, (b) Tier 1 SDKs + spend caps, (c) skip coding until H100s arrive. Answer: (b).
  7. Short Answer: Where should API keys live? Answer: Environment / secret manager—never in git.
  8. True/False: An API-only laptop running FastAPI is still Tier 1. Answer: True (app local, weights remote).
  9. Multiple Choice: Air-gapped hospital LLM: (a) stay Tier 1, (b) leave for local/VPC GPU tiers, (c) email weights to Gmail. Answer: (b).
  10. Short Answer: What is the next tier when you want a local quantized 7B on a 3060/4060? Answer: Tier 2 Local Basic.

Key Takeaways

  • Tier 1 = any laptop + internet + API key; weights stay at the provider.
  • This is the correct default until privacy, TCO, latency, or customization force silicon.
  • Ops are keys, spend, and 18.2 observability—not nvidia-smi.
  • Vol. 06/12 still explain the bill; you just do not own the GPUs.
  • Continue with Tier 2 Local Basic.
Trainer’s Guide

Lab: Each student runs the SDK snippet with a spend-capped key, logs usage, and writes a one-page “stay vs leave Tier 1” memo for a fictional product (consumer homework helper vs hospital note summarizer vs factory offline kiosk).

Discussion: Is a student MacBook running a 3B GGUF on CPU still Tier 1? Force the definition: local weights vs API-only.

Recap: Tier 1 is API-only on any laptop. Continue with Tier 2 Local Basic.