← Master Index
Vol. 07 Module 7.1 Lecture

Average Pooling

CNN Core Concepts

How This Lesson Fits the Module

Max pooling keeps the peak of each window; average pooling keeps the mean. It smooths rather than sharpens, and its global form (Global Average Pooling) is a cornerstone of modern classifier heads—directly relevant to the flatten alternative and transfer learning lectures ahead.

Learning Objectives

By the end of this lesson, students should be able to:

  • Define average pooling and compute its output by hand.
  • Contrast the smoothing effect of averaging with the peak-picking of max.
  • Explain Global Average Pooling (GAP) and why it replaced large dense heads.
  • Describe how gradients distribute across an average-pool window.
  • Apply nn.AvgPool2d and nn.AdaptiveAvgPool2d in PyTorch.
  • Select average vs. max pooling for a given goal.
Definition

Average pooling slides a window over each feature map and outputs the arithmetic mean of the values in each window.

Worked Example

Input (4x4) 2x2 windows -> mean 1 3 | 2 4 (1+3+5+6)/4=3.75 (2+4+7+8)/4=5.25 5 6 | 7 8 ------+------ Output (2x2) 9 2 | 1 0 (9+2+4+3)/4=4.5 (1+0+2+1)/4=1.0 4 3 | 2 1 [3.75 5.25] [4.50 1.00]

Compare with the same input under max pooling (which gave [[6,8],[9,2]]): averaging pulls values toward the middle and never lets a single peak dominate.

Global Average Pooling (GAP)

GAP averages each entire feature map to a single number, turning a (N, C, H, W) tensor into (N, C, 1, 1). Introduced in Network-in-Network and used by ResNet/Inception, it replaces the huge flatten-then-dense head with a parameter-free summary—drastically reducing overfitting and parameter count. It is the standard bridge from convolutional features to the final classifier.

Gradient Distribution

Because the output is a mean, the gradient of the loss splits evenly across all positions in the window (each receives 1/(k·k) of the upstream gradient). Unlike max pooling’s single-position routing, every input contributes and every input learns.

Average Pooling in PyTorch

import torch from torch import nn x = torch.randn(1, 512, 7, 7) # e.g. ResNet final feature maps avg = nn.AvgPool2d(kernel_size=2, stride=2) print(avg(x).shape) # torch.Size([1, 512, 3, 3]) # Global Average Pooling: one value per channel gap = nn.AdaptiveAvgPool2d((1, 1)) pooled = gap(x) print(pooled.shape) # torch.Size([1, 512, 1, 1]) head = nn.Linear(512, 1000) logits = head(torch.flatten(pooled, 1)) print(logits.shape) # torch.Size([1, 1000])

Max vs. Average: Choosing

Use Max When

  • Detecting sharp features.
  • Presence matters more than context.
  • In hidden downsampling layers.

Use Average When

  • Summarizing whole maps (GAP).
  • Smooth, global context helps.
  • Reducing head parameters.

Shared Traits

  • No learnable parameters.
  • Reduce spatial size only.
  • Cheap and deterministic.

Strengths and Tradeoffs

Strengths

  • Retains overall intensity/context.
  • Robust to single noisy activations.
  • GAP slashes head parameters.

Tradeoffs

  • Can dilute a small but important peak.
  • Less effective for crisp feature detection.
  • Averages background into the signal.
Common Misconception

“Average pooling is always weaker than max.” For hidden feature detection max often wins, but Global Average Pooling is superior for classifier heads—fewer parameters, better generalization. The right choice depends on the role, not a universal ranking.

Knowledge Check

  1. Short Answer: What does average pooling output per window? Answer: The arithmetic mean of the window’s values.
  2. True/False: Average pooling is more robust to a single outlier than max pooling. Answer: True.
  3. Multiple Choice: Mean of (2,4,7,8) is: (a) 8, (b) 5.25, (c) 21. Answer: (b).
  4. Short Answer: What does Global Average Pooling produce? Answer: One value per channel, shape (N, C, 1, 1).
  5. True/False: GAP has learnable parameters. Answer: False.
  6. Multiple Choice: In backprop, an avg-pool window distributes gradient: (a) to the max only, (b) evenly to all positions, (c) to none. Answer: (b).
  7. Short Answer: Why did GAP replace large dense heads? Answer: It removes millions of parameters and reduces overfitting.
  8. Short Answer: Which PyTorch layer gives GAP to 1×1? Answer: nn.AdaptiveAvgPool2d((1,1)).
  9. True/False: Average pooling changes the channel count. Answer: False.
  10. Multiple Choice: For crisp feature-presence detection, prefer: (a) average pooling, (b) max pooling, (c) no pooling. Answer: (b).

Key Takeaways

  • Average pooling outputs the mean of each window—smoothing rather than peak-picking.
  • Gradients spread evenly across the window; there are no parameters.
  • Global Average Pooling replaces bulky dense heads and curbs overfitting.
  • Choose max for feature presence, average/GAP for global summaries.
  • Next, Padding shows how to control spatial size at the borders.
Trainer’s Guide

Hands-on idea: Pool one feature map with both methods and overlay the results; discuss which preserved the object and which smoothed the background.

Discussion prompt: Why does Global Average Pooling generalize better than a flatten-plus-dense head?

Recap: Average pooling smooths windows into their mean; its global form is the modern classifier bridge. Continue with Padding.