Transfer Learning — Module 7.1

How This Lesson Fits the Module

Module 7.1 built CNNs from first principles—convolution, kernels, filters, feature maps, pooling, padding, stride, and flatten. Transfer learning is the capstone: instead of training a CNN from scratch, you reuse filters a large model already learned. It ties the whole module together and is how most real computer-vision projects actually begin.

Learning Objectives

By the end of this lesson, students should be able to:

Define transfer learning and explain why pretrained CNN features transfer.
Distinguish feature extraction from fine-tuning.
Decide which layers to freeze based on dataset size and similarity.
Replace and retrain a pretrained model’s classifier head in PyTorch.
Apply the correct input preprocessing for a pretrained backbone.
Avoid common pitfalls: wrong normalization, too-high learning rate, unfrozen BatchNorm.

Definition

Transfer learning reuses a model trained on a large source dataset (e.g., ImageNet) as the starting point for a new, usually smaller, target task—keeping most learned filters and retraining only part of the network.

Why Features Transfer

Recall from the feature map lecture that early CNN layers learn generic edges and textures, while deep layers learn task-specific parts. Edges and textures are useful for almost any image task, so the early filters of an ImageNet model are excellent for medical scans, satellite imagery, or product photos. You keep that reusable knowledge and only relearn the task-specific top.

Two Strategies

Feature Extraction

Freeze the backbone.
Train only a new head.
Fast; best for small data.

Fine-Tuning

Unfreeze some/all layers.
Train with a low learning rate.
Higher accuracy on larger data.

From Scratch

No pretrained weights.
Needs large data + compute.
Rarely the first choice.

A Decision Guide

Target data	Similar to source?	Recommended approach
Small	Yes	Feature extraction (freeze backbone)
Small	No	Feature extraction from earlier layers; careful fine-tuning
Large	Yes	Fine-tune the whole network (low LR)
Large	No	Fine-tune all layers, or consider training from scratch

Feature Extraction in PyTorch

Load a pretrained ResNet, freeze it, and swap the final fc layer for your class count.

import torch
from torch import nn
from torchvision import models

# 1. Load pretrained backbone
model = models.resnet18(weights=models.ResNet18_Weights.DEFAULT)

# 2. Freeze all backbone parameters (feature extraction)
for param in model.parameters():
    param.requires_grad = False

# 3. Replace the classifier head for a 5-class task
num_features = model.fc.in_features    # 512 for resnet18
model.fc = nn.Linear(num_features, 5)  # new head is trainable by default

# 4. Optimize ONLY the new head
optimizer = torch.optim.Adam(model.fc.parameters(), lr=1e-3)
criterion = nn.CrossEntropyLoss()

Fine-Tuning in PyTorch

Unfreeze the later blocks and train the whole model with a small learning rate so pretrained weights are nudged, not destroyed.

for param in model.parameters():
    param.requires_grad = True   # unfreeze everything

# Small LR protects transferred features; consider layer-wise LRs
optimizer = torch.optim.Adam(model.parameters(), lr=1e-5)

# Use the SAME preprocessing the backbone was trained with
weights = models.ResNet18_Weights.DEFAULT
preprocess = weights.transforms()   # resize, center-crop, ImageNet normalize
# transforms(): mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225]

Common Mistakes

Wrong normalization: a model pretrained on ImageNet expects ImageNet mean/std—feeding raw [0,1] pixels quietly wrecks accuracy. Learning rate too high: a large LR during fine-tuning erases the very features you wanted to keep. Forgotten BatchNorm: in feature extraction, keep the backbone in eval() so BatchNorm running stats are not overwritten by tiny target batches.

Why Transfer Learning Wins

Advantages

Works with small datasets.
Trains far faster than from scratch.
Often higher accuracy.

Caveats

Must match input preprocessing.
Domain gap can limit gains.
Large backbones may need quantization to deploy.

Module 7.1 Recap Core op ← convolution, kernel, filters · Output ← feature maps · Downsample ← pooling, max, average · Geometry ← padding, stride · Head ← flatten · Reuse → transfer learning.

Knowledge Check

Short Answer: What is transfer learning? Answer: Reusing a model pretrained on a large dataset as the starting point for a new task.
True/False: Early CNN layers learn generic features that transfer well. Answer: True.
Multiple Choice: Feature extraction means: (a) train everything, (b) freeze backbone and train a new head, (c) delete conv layers. Answer: (b).
Short Answer: When is fine-tuning preferred over feature extraction? Answer: When you have a larger target dataset (and want higher accuracy).
True/False: You can safely skip the backbone’s original normalization. Answer: False—you must match its preprocessing.
Multiple Choice: A good fine-tuning learning rate is: (a) very high, (b) very low, (c) exactly the pretraining LR. Answer: (b).
Short Answer: In PyTorch, how do you freeze a parameter? Answer: Set param.requires_grad = False.
Short Answer: What must you replace when adapting a pretrained classifier? Answer: The final head/fc layer to output the new number of classes.
True/False: Transfer learning usually needs less data than training from scratch. Answer: True.
Multiple Choice: A frequent transfer-learning bug is: (a) wrong input normalization, (b) too few epochs only, (c) using Adam. Answer: (a).

Key Takeaways

Transfer learning reuses pretrained CNN features instead of training from scratch.
Feature extraction freezes the backbone; fine-tuning updates it with a low LR.
Pick a strategy from dataset size and similarity to the source domain.
Always match the backbone’s preprocessing and guard BatchNorm stats.
This completes Module 7.1—you can now build and adapt CNNs end to end.

Trainer’s Guide

Capstone idea: Have students fine-tune a pretrained ResNet-18 on a small 3–5 class image set and report accuracy vs. a from-scratch baseline.

Discussion prompt: Trace the full Module 7.1 pipeline from raw image to prediction—where did each concept (convolution, pooling, padding, stride, flatten) appear?

Module 7.1 Complete You can explain and build convolutional networks, control their geometry, connect them to a classifier, and adapt pretrained models to new tasks with transfer learning.