Pooling

How This Lesson Fits the Module

Convolution produces feature maps that are often as large as the input—expensive and sensitive to tiny shifts. Pooling downsamples them, shrinking spatial size while keeping the strongest signals. This lecture is the umbrella concept for the two variants you will study next: max pooling and average pooling.

Learning Objectives

By the end of this lesson, students should be able to:

Define pooling and state its two main purposes.
Compute a pooled output size from window and stride.
Contrast pooling with strided convolution as downsampling strategies.
Explain how pooling contributes to approximate translation invariance.
Apply nn.MaxPool2d, nn.AvgPool2d, and adaptive pooling in PyTorch.
Recognize when to reduce or omit pooling in modern architectures.

Definition

Pooling slides a small window over each feature map and replaces every window with a single summary value (its maximum or average), reducing spatial resolution while preserving channel count.

Why Downsample?

Pooling serves two goals. First, efficiency: halving height and width cuts the activation size (and downstream compute) by 4×. Second, robustness: summarizing a neighborhood means a small pixel shift rarely changes the pooled value, giving the network approximate translation invariance—the property convolution alone lacks.

Aspect	Before pooling	After 2×2 pool
Spatial size	32 × 32	16 × 16
Channels	C (unchanged)	C (unchanged)
Activations	1024 · C	256 · C
Learnable params	—	0 (pooling has none)

Output-Size Formula

For window k and stride s (default s = k): W_out = floor((W - k)/s) + 1. A 2×2 pool with stride 2 halves each spatial dimension.

Pooling vs. Strided Convolution

No learnable parameters.
Fixed summary (max/avg).
Cheap, deterministic.

Strided Conv

Learns how to downsample.
Adds parameters.
Common in modern nets (ResNet).

Adaptive Pool

Targets a fixed output size.
Decouples head from input size.
Used before classifiers.

Pooling in PyTorch

import torch
from torch import nn

x = torch.randn(1, 8, 32, 32)

maxpool = nn.MaxPool2d(kernel_size=2, stride=2)
avgpool = nn.AvgPool2d(kernel_size=2, stride=2)
print(maxpool(x).shape)   # torch.Size([1, 8, 16, 16])
print(avgpool(x).shape)   # torch.Size([1, 8, 16, 16])

# Global/adaptive pooling to a fixed 1x1 before the classifier head
gap = nn.AdaptiveAvgPool2d((1, 1))
print(gap(x).shape)       # torch.Size([1, 8, 1, 1])

Common Misconception

“Pooling changes the number of channels.” It does not. Pooling operates on each channel independently and reduces only spatial dimensions. Channel count is set by filters, not pooling.

Are We Still Pooling?

Classic nets (LeNet, AlexNet, VGG) pooled aggressively. Some modern designs prefer strided convolutions for downsampling and keep only a final global average pool before the classifier. Pooling is still standard, but treat it as one downsampling tool among several.

Knowledge Check

Short Answer: What does pooling do? Answer: Summarizes each window into one value, reducing spatial size.
True/False: Pooling reduces the number of channels. Answer: False—it reduces only spatial dimensions.
Multiple Choice: A 2×2 pool with stride 2 on 32×32 gives: (a) 32×32, (b) 16×16, (c) 8×8. Answer: (b).
Short Answer: Name pooling’s two main benefits. Answer: Fewer activations/compute and approximate translation invariance.
True/False: Pooling layers have learnable parameters. Answer: False.
Multiple Choice: An alternative to pooling for downsampling is: (a) dropout, (b) strided convolution, (c) softmax. Answer: (b).
Short Answer: What does AdaptiveAvgPool2d((1,1)) produce? Answer: One value per channel (a global average), shape (N, C, 1, 1).
Short Answer: The two common pooling operations are? Answer: Max pooling and average pooling.
True/False: Larger pooling windows discard more spatial detail. Answer: True.
Multiple Choice: Pooling helps invariance because it: (a) learns weights, (b) summarizes a neighborhood, (c) adds channels. Answer: (b).

Key Takeaways

Pooling downsamples feature maps spatially while leaving channels untouched.
It reduces compute and grants approximate translation invariance—with zero parameters.
Output size follows floor((W - k)/s) + 1; 2×2 stride-2 halves dimensions.
Strided convolutions and global pooling are common modern alternatives/companions.
Next, Max Pooling covers the most widely used variant.

Trainer’s Guide

Hands-on idea: Pool the same feature map with max and average, then compare which preserves sharp features versus smooth backgrounds.

Discussion prompt: Why might a modern architecture replace most pooling with strided convolutions?

Recap: Pooling shrinks feature maps and adds shift-robustness at no parameter cost. Continue with Max Pooling.