← Master Index
Vol. 07 Module 7.1 Lecture

Padding

CNN Core Concepts

How This Lesson Fits the Module

Convolution shrinks feature maps at the borders and undersamples edge pixels. Padding adds a border around the input so you can control the output size and treat edges fairly. Together with stride (next lecture), padding is one of the two knobs that determine every feature map’s spatial dimensions.

Learning Objectives

By the end of this lesson, students should be able to:

  • Define padding and explain the two problems it solves.
  • Distinguish “valid” (no padding) from “same” padding.
  • Compute the padding needed to preserve spatial size for a given kernel.
  • Compare zero, reflection, and replication padding modes.
  • Configure padding in nn.Conv2d and verify output shapes.
  • Reason about how padding interacts with the output-size formula.
Definition

Padding adds extra rows and columns (commonly zeros) around the border of an input before convolution, so the kernel can be centered on edge pixels and the output size can be controlled.

Two Problems Padding Solves

Shrinking: a 3×3 kernel with no padding turns a 32×32 input into 30×30; stack many such layers and the map vanishes. Border neglect: without padding, corner pixels are covered by far fewer kernel positions than central pixels, so edge information is underused. Padding fixes both.

Input 3x3 Zero-padded (pad=1) 5x5 1 2 3 0 0 0 0 0 4 5 6 ---> 0 1 2 3 0 7 8 9 0 4 5 6 0 0 7 8 9 0 0 0 0 0 0 A 3x3 kernel on the padded input yields a 3x3 output (size preserved).

Valid vs. Same

ModePaddingOutput size (k, stride 1)Effect
Valid0W - k + 1Shrinks each layer
Same(k-1)/2WPreserves size
Same-Padding Rule

For an odd kernel k with stride 1, set p = (k - 1) / 2: k=3→p=1, k=5→p=2, k=7→p=3. Recall the full formula: W_out = floor((W - k + 2p)/s) + 1.

Padding Modes

Zero

  • Pads with 0s.
  • Default and simplest.
  • Can introduce a dark border bias.

Reflection

  • Mirrors edge pixels.
  • Avoids hard zero edges.
  • Popular in style transfer.

Replication

  • Repeats the edge value.
  • Smooth border continuation.
  • Common in segmentation.

Padding in PyTorch

import torch from torch import nn x = torch.randn(1, 3, 32, 32) valid = nn.Conv2d(3, 8, kernel_size=3, padding=0) # "valid" same = nn.Conv2d(3, 8, kernel_size=3, padding=1) # "same" (stride 1) print(valid(x).shape) # torch.Size([1, 8, 30, 30]) print(same(x).shape) # torch.Size([1, 8, 32, 32]) # String form and non-zero modes same_str = nn.Conv2d(3, 8, 3, padding="same") reflect = nn.Conv2d(3, 8, 3, padding=1, padding_mode="reflect") print(same_str(x).shape, reflect(x).shape)
Common Misconception

“Zero padding adds meaningful data.” The padded zeros carry no signal; they exist only to control geometry and let the kernel sit on border pixels. Over-padding can even inject a faint border artifact—prefer just enough for your target size.

Knowledge Check

  1. Short Answer: What is padding? Answer: Extra border rows/columns added to the input before convolution.
  2. True/False: “Valid” padding preserves spatial size. Answer: False—valid means no padding, so size shrinks.
  3. Multiple Choice: Same padding for a 5×5 kernel (stride 1) is: (a) 1, (b) 2, (c) 4. Answer: (b).
  4. Short Answer: Name the two problems padding addresses. Answer: Shrinking feature maps and underuse of border pixels.
  5. True/False: Zero padding adds informative pixel values. Answer: False—the values are zeros with no signal.
  6. Multiple Choice: A 32×32 input, 3×3 kernel, padding 0, stride 1 gives: (a) 32, (b) 30, (c) 34. Answer: (b).
  7. Short Answer: Which padding mode mirrors edge pixels? Answer: Reflection padding.
  8. Short Answer: Give the general output-size formula. Answer: floor((W - k + 2p)/s) + 1.
  9. True/False: With padding="same" and stride 1, output height equals input height. Answer: True.
  10. Multiple Choice: Same padding for k=7 (stride 1) is: (a) 2, (b) 3, (c) 6. Answer: (b).

Key Takeaways

  • Padding adds a border so convolutions can preserve size and use edge pixels.
  • Valid = no padding (shrinks); Same = (k-1)/2 padding (preserves size at stride 1).
  • Zero, reflection, and replication modes trade simplicity for smoother borders.
  • Padding p enters the output-size formula as +2p.
  • Next, Stride is the other knob controlling output size.
Trainer’s Guide

Hands-on idea: Have students predict output shapes for padding 0, 1, and 2 on a fixed kernel, then confirm with nn.Conv2d.

Discussion prompt: When might reflection or replication padding beat zero padding?

Recap: Padding controls border handling and output size; “same” padding keeps dimensions intact. Continue with Stride.