GPT-3 base models complete text; products need helpful, harmless, honest assistants. GPT-3.5 names the generation of models (including ChatGPT-era systems) sharpened with instruction tuning and preference optimization / RLHF-style alignment—bridging raw LLMs to conversational UX before GPT-4.
Deep alignment mechanics live in Module 11.4; here we focus on the product/capability step in the GPT timeline.
Learning Objectives
By the end of this lesson, students should be able to:
- Position GPT-3.5 between GPT-3 and GPT-4 on the timeline.
- Contrast base completion models with instruction-following chat models.
- Describe the role of instruction data and human preference feedback at a high level.
- Explain chat formatting (system / user / assistant turns) vs. raw prompts.
- Relate latency/cost tiers (e.g., turbo-class serving) to engineering tradeoffs.
- Avoid treating marketing names as precise architecture papers.
GPT-3.5 refers to the post–GPT-3 generation of OpenAI models optimized for dialogue and instruction following—typically via supervised instruction tuning and reinforcement learning from human feedback (or related preference methods)—rather than raw unsupervised completion alone.
Timeline
Few-shot base LM.
Follow instructions; chat.
Mass consumer UX.
Stronger multimodal reasoner.
Base GPT-3 style
- Continues any text
- Prompt engineering heavy
- Can ignore “be helpful”
GPT-3.5 chat style
- Trained to answer users
- Multi-turn dialogue
- Safer defaults (imperfect)
Still underneath
- Autoregressive decoding
- Sampling / temperature
- Context window limits
What Changed for Engineers
| Practice | Before (raw LM) | After (3.5-style chat) |
|---|---|---|
| Interface | Single completion string | Messages with roles |
| Control | Prompt only | System + user messages |
| Behavior | Often needs examples | Better zero-shot instructions |
| Risks | Unfiltered completion | Reduced but not eliminated |
“GPT-3.5 is a brand-new neural architecture unrelated to GPT-3.” Public descriptions emphasize training/alignment and productization on the generative LM stack—not a jump to a non-Transformer paradigm.
Strengths and Tradeoffs
Strengths
- Dramatically better instruction following.
- Natural multi-turn assistants.
- Enabled mainstream LLM apps.
Tradeoffs
- Alignment can over-refuse or sycophantically agree.
- Closed details; hard to reproduce exactly.
- Still hallucinates confidently.
Knowledge Check
- Short Answer: Where does GPT-3.5 sit on the timeline? Answer: After GPT-3, before GPT-4.
- True/False: GPT-3.5-era models are mainly about chat/instruction alignment. Answer: True.
- Multiple Choice: RLHF-style methods: (a) shape behavior with preferences, (b) replace attention with convolutions only, (c) delete the tokenizer. Answer: (a).
- Short Answer: Name three common chat roles. Answer: System, user, assistant.
- True/False: Aligned chat models never hallucinate. Answer: False.
- Multiple Choice: Raw GPT-3 completion vs. 3.5 chat: (a) chat follows instructions better on average, (b) identical UX, (c) chat cannot use prompts. Answer: (a).
- Short Answer: Which later module covers RLHF in depth? Answer: Module 11.4.
- Short Answer: Why did ChatGPT feel different from playground completion? Answer: Dialogue-oriented instruction/preference training and UX.
- Multiple Choice: Under the hood, decoding is still: (a) autoregressive token generation, (b) only k-NN search, (c) JPEG compression. Answer: (a).
- True/False: Marketing model names always equal detailed open architecture papers. Answer: False.
Key Takeaways
- GPT-3.5 marks the instruction/chat alignment era.
- Messages + roles replace raw string completion UX.
- Preferences/RLHF-style training steer behavior.
- Still an autoregressive GPT-family model underneath.
- Next: GPT-4.
Hands-on idea: Compare the same ask as a raw completion prompt vs. a system+user chat transcript on any available model.
Discussion prompt: How can over-alignment hurt legitimate technical questions?
Recap: GPT-3.5 turned base LMs into instruction-following assistants. Continue with GPT-4.