VGG16 — Module 7.3

How This Lesson Fits the Module

AlexNet used a mix of large kernels (11×11, 5×5). VGG (Simonyan & Zisserman, Oxford, 2014) asked a cleaner question: what if we use only 3×3 convolutions and simply go deeper? The answer—a strikingly uniform, easy-to-understand architecture—became the go-to feature extractor for years and a template for principled depth.

AlexNet (2012) — mixed 11×11 / 5×5 / 3×3 kernels, 8 layers ↓ VGG16 (2014) — uniform 3×3 convs, 16 layers, depth over kernel size ↓ Next: ResNet solves the depth ceiling with skip connections

Learning Objectives

By the end of this lesson, students should be able to:

Explain VGG’s core design principle: stacked 3×3 convolutions.
Show why two 3×3 convs match one 5×5 receptive field with fewer parameters.
Describe the VGG16 block structure and its ~138M parameters.
Load and adapt VGG16 from torchvision.models.
Explain VGG’s strengths (simplicity, transferable features) and weaknesses (size, cost).
Understand why depth eventually hit a wall that ResNet had to break.

Definition — VGG16

VGG16 is a 16-weight-layer CNN (13 convolutional + 3 fully connected) built entirely from 3×3 convolutions and 2×2 max pooling. Channels double after each pooling stage (64 → 128 → 256 → 512), and the network ends in three dense layers and a 1000-way softmax.

The Key Insight: Small Kernels, Deep Stacks

Two stacked 3×3 convolutions have the same 5×5 receptive field as one 5×5 conv—but use fewer parameters and add an extra ReLU, giving more nonlinearity. Three stacked 3×3 convs match a 7×7 field. This is why VGG replaces big kernels with deep stacks of tiny ones.

Choice	One 5×5 conv	Two 3×3 convs
Receptive field	5×5	5×5
Params (C in/out)	25 C²	18 C²
Nonlinearities	1 ReLU	2 ReLU
Expressiveness	Lower	Higher

VGG16 Architecture

Block	Layers	Output channels
1	2 × conv3-64, maxpool	64
2	2 × conv3-128, maxpool	128
3	3 × conv3-256, maxpool	256
4	3 × conv3-512, maxpool	512
5	3 × conv3-512, maxpool	512
Head	FC-4096, FC-4096, FC-1000	1000

Total: ~138 million parameters, the bulk in the first FC layer (7×7×512 → 4096). VGG16 is accurate and transfers beautifully—but heavy.

Loading VGG16 in PyTorch

import torch
import torch.nn as nn
from torchvision.models import vgg16, VGG16_Weights

weights = VGG16_Weights.IMAGENET1K_V1
model = vgg16(weights=weights)

# Use VGG as a fixed feature extractor for a 5-class problem
for p in model.features.parameters():
    p.requires_grad = False
model.classifier[6] = nn.Linear(4096, 5)

n_params = sum(p.numel() for p in model.parameters())
print(f"VGG16 parameters: {n_params:,}")     # ~138,357,544

# A single VGG "conv block" you could build yourself:
def vgg_block(in_ch, out_ch, n_convs):
    layers = []
    for _ in range(n_convs):
        layers += [nn.Conv2d(in_ch, out_ch, 3, padding=1), nn.ReLU(inplace=True)]
        in_ch = out_ch
    layers.append(nn.MaxPool2d(2))
    return nn.Sequential(*layers)

Module 7.1 Tie-In Every conv uses padding=1 to preserve spatial size within a block; downsampling happens only at max pooling. Doubling channels while halving resolution is the classic CNN trade.

VGG Strengths

Simple, uniform, easy to reason about
Excellent transferable features
Strong ImageNet accuracy for its era
Great teaching architecture

VGG Limitations

~138M params—memory heavy
Slow inference vs modern nets
Plain depth saturates accuracy
Giant FC layers dominate size

Common Misconception: “Deeper VGG always means better accuracy.”

Reality: Beyond ~19 layers, plain stacks like VGG stop improving and can degrade—the vanishing-gradient / optimization wall. Solving that required residual connections, which is exactly what ResNet introduced in 2015.

Critical Mistake — Deploying VGG on Edge Devices

VGG16’s size and FLOPs make it a poor fit for phones or embedded systems. If latency and memory matter, reach for MobileNet or EfficientNet instead. VGG is best as a research baseline or feature extractor on a server.

Knowledge Check

Short Answer: What single kernel size does VGG use throughout its conv layers? Answer: 3×3.
True/False: Two stacked 3×3 convs have the same receptive field as one 5×5 conv. Answer: True.
Multiple Choice: VGG16 has how many weight layers? (a) 8, (b) 13, (c) 16, (d) 50. Answer: (c).
Short Answer: Roughly how many parameters does VGG16 have? Answer: About 138 million.
True/False: Channels double after each pooling stage in VGG. Answer: True (64→128→256→512).
Multiple Choice: Most of VGG16’s parameters live in: (a) first conv, (b) pooling, (c) fully connected layers, (d) softmax. Answer: (c).
Short Answer: Give one advantage of stacking 3×3 convs over one large kernel. Answer: Fewer params and more nonlinearity for the same receptive field.
True/False: VGG is a great choice for low-latency mobile deployment. Answer: False.
Multiple Choice: Plain depth in VGG saturates because of: (a) too few classes, (b) optimization / vanishing gradients, (c) softmax, (d) dropout. Answer: (b).
Short Answer: Which 2015 model broke the plain-depth ceiling? Answer: ResNet.

Key Takeaways

VGG16 uses only 3×3 convs and 2×2 max pooling in a uniform, deep stack.
Stacked small kernels match large receptive fields with fewer params and more ReLUs.
~138M parameters make it accurate but heavy and slow.
Plain depth saturates—motivating residual connections.
Next: ResNet adds skip connections to train 50–152 layers.

Trainer’s Guide

Exercise: Have students compute the parameter count of one 7×7 conv vs three stacked 3×3 convs (same channels) and confirm VGG’s efficiency argument.

Whiteboard: Draw the 5 VGG blocks and annotate spatial size halving / channel doubling—this pattern recurs in nearly every backbone.

Progress LeNet → AlexNet → VGG. We have pushed depth to its plain-network limit. Continue to ResNet to see how skip connections shatter that ceiling.