Module 11.3 covered numbered GPT milestones. This lecture treats the GPT family as a product lineage: decoder-only transformers, scaled pretraining, and post-training for chat/tools. It opens Module 11.5 after Context Length.
Learning Objectives
By the end of this lesson, students should be able to:
- Place GPT models in the decoder-only causal LM lineage.
- Separate base completion models from ChatGPT-style assistants.
- Describe closed API access versus open-weight alternatives.
- List typical surfaces: chat, APIs, agents, multimodal.
- State trade-offs: capability, cost, opacity, lock-in.
- Avoid fabricating unpublished proprietary internals.
The GPT family (Generative Pre-trained Transformer) is OpenAI’s lineage of primarily decoder-only language models trained with next-token prediction and adapted with instruction/preference methods into widely deployed assistants and APIs.
Architecture & Lineage
| Stage / model | Publicly known shift | Curriculum note |
|---|---|---|
| GPT-1 / GPT-2 | Unsupervised pretrain + generation demos | Historical teaching baselines |
| GPT-3 | In-context learning at API scale | Few-shot prompting era |
| Instruct / ChatGPT era | SFT + preference alignment | Productized assistants |
| GPT-4 class & successors | Stronger reasoning/tools/multimodal SKUs | Closed flagship tier |
Causal LM at scale.
SFT + preferences.
Chat, APIs, tools.
New SKUs & modalities.
Open vs Closed Positioning
Access
- Mostly closed API / product access
- Flagship weights not generally public
- Contrast with Llama / Mistral open weights
Differentiation
- Strong general assistant ecosystem
- Tools, multimodal, enterprise controls
- Fast product iteration
Curriculum links
- Builds on Module 11.3 GPT lectures
- Compare next with Claude & Gemini
- Open peers: Llama, Mistral, Qwen
Typical Use Cases
| Use case | Why this family | Watch-outs |
|---|---|---|
| General assistants | Polished chat + tool ecosystems | Vendor cost & data policies |
| App backends | Stable APIs and managed scale | Rate limits; provider dependency |
| Multimodal features | Image+text product paths | Opaque internals vs open forks |
Engineering Touchpoint
Engineering Upsides
- High capability for many tasks
- Mature SDKs and enterprise features
- Lower ops burden than self-hosting
Engineering Trade-offs
- Closed weights limit deep customization
- Pricing/policy can change
- Hard to audit model internals
“GPT means one frozen architecture.” GPT is a brand lineage; generations change scale, data, post-training, and multimodal packaging. Treat SKU names as products, not eternal papers.
Knowledge Check
- Short Answer: What does GPT stand for? Answer: Generative Pre-trained Transformer.
- True/False: Flagship GPT models are typically fully open weights. Answer: False—they are primarily closed APIs/products.
- Multiple Choice: GPT models are primarily: (a) encoder-only BERT clones, (b) decoder-only causal LMs, (c) pure CNNs, (d) random forests. Answer: (b).
- Short Answer: What objective started the GPT lineage? Answer: Next-token / autoregressive language modeling.
- True/False: ChatGPT-style behavior needs no post-training. Answer: False—SFT and preference methods matter.
- Multiple Choice: A key GPT API trade-off is: (a) free infinite tokens, (b) capability vs cost/lock-in, (c) mandatory open weights, (d) no tokenizer. Answer: (b).
- Short Answer: Name one enterprise reason to use GPT APIs. Answer: Strong quality, tooling, and managed infrastructure.
- True/False: Module 11.3 already covered earlier GPT milestones. Answer: True.
- Multiple Choice: Versus Llama flagships, GPT flagships are usually: (a) more closed, (b) always smaller, (c) CNNs, (d) non-neural. Answer: (a).
- Short Answer: Why avoid inventing GPT internals? Answer: Proprietary details are unpublished; stick to public docs and engineering facts.
Key Takeaways
- GPT is OpenAI’s decoder-only LM lineage turned product platform.
- Access is mostly closed; ecosystem and quality are the draw.
- Post-training and tools matter as much as raw scale.
- Next: Claude Family.
Compare: Same coding prompt on a GPT API vs an open 7–8B instruct model.
Policy: Extract claims vs unknowns from a public system/model card.
Recap: The GPT family anchors closed, productized decoder-only LLMs. Continue with Claude Family.