You just defined the LLM. Foundation model is the broader industry term: a model pretrained at scale so it can be adapted to many downstream tasks. LLMs are the text (and often multimodal) instance of that idea.
This framing matters for product strategy: you rarely train from scratch; you choose a foundation model, then apply instruction tuning, fine-tuning, or RAG. Module 11.5’s families are competing foundation offerings.
Learning Objectives
By the end of this lesson, students should be able to:
- Define foundation model and relate it to LLMs and multimodal models.
- Explain adaptation: prompting, SFT, PEFT, and full fine-tuning.
- Contrast foundation models with single-task trained models.
- List selection criteria: license, modality, context, cost, evals.
- Describe the foundation → adapter → application stack.
- Connect the term to open vs closed ecosystems in Module 11.5.
A foundation model is a model trained on broad data at scale such that it can be adapted (prompted, fine-tuned, or composed with tools) to a wide range of downstream tasks. The Stanford CRFM usage popularized the term for models that serve as a base platform for many applications.
Foundation vs Task-Specific
| Aspect | Task-specific model | Foundation model |
|---|---|---|
| Training goal | One labeled task | Broad self-supervised (or multi-task) pretraining |
| Reuse | Limited | Many products share one base |
| Adaptation | Retrain / new head often | Prompt, SFT, PEFT, tools |
| Risk | Narrow failure modes | Shared failures propagate widely |
Adaptation Ladder
No weight updates; inject context.
Train tiny adapters (LoRA, etc.).
Update many/all weights.
Domain corpus, still self-supervised.
Modalities
Text LLM
- Chat, coding, documents.
- Core of Module 11.
Vision / Speech
- Image/video/audio encoders or natives.
- Often fused into multimodal FMs.
Multimodal FM
- Shared or coupled towers.
- One API for text+image(+more).
Why Teams Use FMs
- Amortize pretraining cost across apps.
- Faster time-to-product via prompting.
- Ecosystem: tools, evals, hosting.
Governance Costs
- Vendor lock-in / license limits.
- Systemic bias and safety issues.
- Harder to certify than narrow models.
“Foundation model means the weights are open and free.” Foundation describes role and training regime, not license. Closed API models and open-weight models can both be foundation models.
Knowledge Check
- Short Answer: What is a foundation model in one sentence? Answer: A broadly pretrained model intended to be adapted to many downstream tasks.
- True/False: Every foundation model is open-source. Answer: False—open vs closed is a licensing/access choice.
- Multiple Choice: The cheapest adaptation step is usually: (a) full fine-tune, (b) prompting/RAG, (c) training from scratch, (d) collecting a new web crawl. Answer: (b).
- Short Answer: Name two adaptation methods that update weights. Answer: Fine-tuning / PEFT (LoRA), and continued pretraining.
- True/False: LLMs are one kind of foundation model. Answer: True.
- Multiple Choice: A shared risk of foundation models is: (a) they never fail, (b) failures can propagate across many apps, (c) they cannot be prompted, (d) they have no parameters. Answer: (b).
- Short Answer: What does “adaptation ladder” mean here? Answer: Increasingly heavy ways to specialize a base FM, from prompting to full FT.
- True/False: Task-specific models never make sense once FMs exist. Answer: False—narrow, auditable models still win in many regulated settings.
- Multiple Choice: Selecting an FM should consider: (a) only logo color, (b) license, modality, cost, evals, (c) only CPU brand, (d) only font choice. Answer: (b).
- Short Answer: Why does the FM framing matter for products? Answer: You build on a shared base platform rather than training every app from scratch.
Key Takeaways
- Foundation models are broadly pretrained bases for many tasks.
- LLMs are the dominant text (and multimodal) foundation stack.
- Adaptation ranges from prompts to PEFT to full fine-tuning.
- Next: Instruction Tuning teaches models to follow user intents.
Exercise: Pick a product (support bot, code helper). List what stays in the FM vs what must be RAG, tools, or fine-tunes.
Debate: When is a small task model better than a general FM?
Recap: Foundation models are reusable pretrained bases; LLMs are the text-centric case. Continue with Instruction Tuning.