Flatten — Module 7.1

How This Lesson Fits the Module

You have built the convolutional stack: convolution, filters, feature maps, pooling, padding, and stride. But the final classifier is a dense layer that expects a 1D vector, not a 3D grid. Flatten is the bridge—it turns spatial feature maps into the vector the head consumes, connecting Volume 07 CNNs back to the dense networks of Volume 06.

Learning Objectives

By the end of this lesson, students should be able to:

Define flatten and state where it appears in a CNN.
Compute the flattened vector length from a feature-map shape.
Explain why the batch dimension must be preserved during flattening.
Contrast flatten-then-dense with Global Average Pooling heads.
Use nn.Flatten, torch.flatten, and view/reshape correctly.
Diagnose the classic shape-mismatch error at the conv-to-dense boundary.

Definition

Flatten reshapes a multi-dimensional feature-map tensor into a 2D tensor of shape (N, C·H·W)—one feature vector per sample—so it can enter a fully-connected (Linear) layer.

Computing the Flattened Length

Flatten collapses the channel and spatial dimensions into one. For a (N, 32, 8, 8) tensor, each sample becomes a vector of length 32 × 8 × 8 = 2048, giving output shape (N, 2048). That number is exactly the in_features your first dense layer must declare.

Feature-map shape (N,C,H,W)	Flattened shape	Vector length
(4, 16, 16, 16)	(4, 4096)	16·16·16 = 4096
(4, 32, 8, 8)	(4, 2048)	32·8·8 = 2048
(4, 64, 4, 4)	(4, 1024)	64·4·4 = 1024

Keep the Batch Dimension

Flatten must fold only the feature dimensions and never the batch dimension—each image needs its own vector. That is why nn.Flatten() defaults to start_dim=1 and torch.flatten(x, 1) starts at dimension 1. Flattening from dimension 0 would merge the whole batch into one giant vector and break training.

Flatten in PyTorch

import torch
from torch import nn

x = torch.randn(4, 32, 8, 8)   # (N, C, H, W)

# Three equivalent ways to flatten (keeping the batch dim)
print(nn.Flatten()(x).shape)         # torch.Size([4, 2048])
print(torch.flatten(x, 1).shape)     # torch.Size([4, 2048])
print(x.view(x.size(0), -1).shape)   # torch.Size([4, 2048])

classifier = nn.Sequential(
    nn.Flatten(),            # (4, 32, 8, 8) -> (4, 2048)
    nn.Linear(32 * 8 * 8, 128),
    nn.ReLU(),
    nn.Linear(128, 10),
)
print(classifier(x).shape)   # torch.Size([4, 10])

Flatten vs. Global Average Pooling

Flatten + Dense

Keeps all spatial detail.
Ties model to a fixed input size.
Large parameter count.

Global Avg Pool

One value per channel.
Works for any input size.
Far fewer parameters.

When to Pick

Flatten: small, fixed inputs.
GAP: modern deep nets.
Both feed a Linear head.

Flatten strengths

Simple and lossless.
Preserves every activation.
Direct path to dense layers.

Watch for

Huge in_features → many parameters.
Fixed input size requirement.
Overfitting without regularization.

Common Mistake

The infamous RuntimeError: mat1 and mat2 shapes cannot be multiplied almost always means the Linear layer’s in_features does not match C·H·W. Print the shape right after flattening and set in_features to that value—or use nn.LazyLinear to infer it automatically.

Knowledge Check

Short Answer: What does flatten do in a CNN? Answer: Reshapes feature maps into a per-sample 1D vector for a dense layer.
True/False: Flatten should merge the batch dimension too. Answer: False—the batch dimension must be preserved.
Multiple Choice: Flattening (N, 64, 4, 4) gives vector length: (a) 256, (b) 1024, (c) 64. Answer: (b) (64·4·4).
Short Answer: What is the default start_dim of nn.Flatten? Answer: 1.
True/False: Global Average Pooling generally uses fewer parameters than flatten+dense. Answer: True.
Multiple Choice: A shape-mismatch RuntimeError at the head usually means: (a) wrong learning rate, (b) mismatched Linear in_features, (c) bad optimizer. Answer: (b).
Short Answer: Give two PyTorch ways to flatten keeping the batch dim. Answer: nn.Flatten() and torch.flatten(x, 1) (or x.view(x.size(0), -1)).
Short Answer: Why does flatten tie the model to a fixed input size? Answer: The flattened length (hence Linear in_features) depends on H and W.
True/False: Flatten has learnable parameters. Answer: False.
Multiple Choice: nn.LazyLinear helps by: (a) skipping flatten, (b) inferring in_features automatically, (c) pooling. Answer: (b).

Key Takeaways

Flatten reshapes (N, C, H, W) feature maps into (N, C·H·W) vectors for the classifier.
Always preserve the batch dimension—flatten from dim 1.
The flattened length equals the Linear layer’s required in_features.
Global Average Pooling is a leaner, size-agnostic alternative.
Next, Transfer Learning reuses pretrained CNNs for new tasks.

Trainer’s Guide

Hands-on idea: Deliberately mis-set a Linear layer’s in_features, trigger the shape error, then fix it by printing the post-flatten shape.

Discussion prompt: Why do many modern architectures prefer Global Average Pooling over flatten+dense?

Recap: Flatten converts spatial feature maps into vectors so dense layers can classify them. Continue with Transfer Learning.