Feature Map — Module 7.1

How This Lesson Fits the Module

A filter convolves over the input and outputs a grid of responses: the feature map. It is the actual data flowing between CNN layers. Understanding feature maps—their shape, meaning, and how activations spread across them—is what lets you interpret and debug a network, and it sets up why we need pooling next.

Learning Objectives

By the end of this lesson, students should be able to:

Define a feature map (activation map) and its role between conv layers.
Interpret the (N, C, H, W) shape of a feature-map tensor.
Explain what high vs. low activation values indicate spatially.
Describe how the receptive field grows across stacked layers.
Extract and visualize intermediate feature maps in PyTorch.
Connect feature maps to the concepts of channels and depth.

Definition

A feature map (or activation map) is the 2D output produced by one filter after convolution (and usually an activation). Each value measures how strongly that filter’s pattern appears at that spatial location.

Reading a Feature-Map Tensor

Inside a CNN, activations are 4D tensors (N, C, H, W): batch, channels, height, width. The C dimension holds one feature map per filter. A convolution turns a (8, 16, 28, 28) tensor into, say, (8, 32, 28, 28)—same spatial size, more channels, each channel a distinct feature map.

Dimension	Symbol	Meaning
Batch	N	Number of images processed together
Channels	C	One feature map per filter
Height	H	Spatial rows of each map
Width	W	Spatial columns of each map

What the Numbers Mean

After ReLU, a feature map is non-negative. A bright spot means the filter’s pattern (say, a diagonal edge) is strongly present at that location; zero means it is absent. Because convolution is equivariant, moving the object in the input moves the bright region in the feature map correspondingly.

Growing Receptive Fields

A single 3×3 conv unit sees a 3×3 patch of the input. Stack another 3×3 layer and each unit now effectively sees 5×5 of the original image; add pooling and it grows faster. Deep feature maps therefore summarize large regions—why late layers can respond to whole objects.

Layer 1

Sees 3×3 — edges.

Layer 2

Sees ~5×5 — corners/textures.

+ Pooling

Receptive field jumps.

Deep layer

Sees most of the image — objects.

Extracting Feature Maps in PyTorch

A forward hook captures an intermediate layer’s output so you can visualize it.

import torch
from torch import nn

conv = nn.Sequential(
    nn.Conv2d(3, 8, 3, padding=1),
    nn.ReLU(),
)

activations = {}
def hook(module, inp, out):
    activations["conv"] = out.detach()

conv[0].register_forward_hook(hook)

x = torch.randn(1, 3, 32, 32)
_ = conv(x)

fmaps = activations["conv"]
print(fmaps.shape)          # torch.Size([1, 8, 32, 32]) -> 8 feature maps
print(fmaps[0, 0].shape)    # torch.Size([32, 32]) -> visualize as grayscale
print(fmaps.mean().item())  # average activation strength

Common Misconception

“Each feature map corresponds to one input channel.” No—each feature map corresponds to one filter, which already blended all input channels. The channel count of a feature-map tensor equals the number of filters in the layer that produced it.

Feature Maps vs. Images

Similarities

Both are spatial 2D grids.
Both can be visualized as heatmaps.
Both preserve position information.

Differences

Values are learned responses, not colors.
There can be dozens or hundreds of channels.
Spatial size usually shrinks with depth.

Knowledge Check

Short Answer: What is a feature map? Answer: The 2D grid of responses produced by one filter, showing where its pattern appears.
True/False: Feature-map channel count equals the number of filters in the producing layer. Answer: True.
Multiple Choice: After ReLU, feature-map values are: (a) always negative, (b) non-negative, (c) between -1 and 1. Answer: (b).
Short Answer: What does a bright region in a feature map indicate? Answer: The filter’s pattern is strongly present at that location.
True/False: Deeper layers have smaller receptive fields than shallow layers. Answer: False—receptive fields grow with depth.
Multiple Choice: The C in an (N, C, H, W) activation tensor is: (a) batch, (b) channels/feature maps, (c) columns. Answer: (b).
Short Answer: Why does moving an object shift the feature-map response? Answer: Because convolution is translation equivariant.
Short Answer: How can you capture an intermediate feature map in PyTorch? Answer: Register a forward hook on the layer.
True/False: A feature map corresponds to a single input channel. Answer: False—it corresponds to a filter that spans all input channels.
Multiple Choice: Stacking conv layers primarily: (a) shrinks channels, (b) grows the receptive field, (c) removes activations. Answer: (b).

Key Takeaways

A feature map is one filter’s spatial response grid; channels stack many together.
Activation tensors are (N, C, H, W); C holds the feature maps.
Bright values mark where a pattern is present; equivariance moves them with the input.
Receptive fields grow with depth, so deep maps summarize larger regions.
Next, Pooling shows how we downsample these maps.

Trainer’s Guide

Hands-on idea: Feed one real image through a pretrained model’s first conv layer and plot all early feature maps as a grid of heatmaps.

Discussion prompt: How could inspecting feature maps help debug a model that ignores part of an image?

Recap: Feature maps are the spatial activations flowing through a CNN, one per filter. Continue with Pooling.