← Master Index
Vol. 07 Module 7.1 Lecture

Kernel

CNN Core Concepts

How This Lesson Fits the Module

Convolution slides a small weight grid over the input. That grid is the kernel. Understanding kernels—their size, values, and effect—is the bridge between the abstract operation and the concrete filters a CNN learns to produce useful feature maps.

Learning Objectives

By the end of this lesson, students should be able to:

  • Define a kernel and distinguish it from the broader term “filter.”
  • Explain how kernel values determine what pattern is detected.
  • Compare common kernel sizes (1×1, 3×3, 5×5, 7×7) and their tradeoffs.
  • Hand-craft classic kernels (edge, blur, sharpen) and predict their effect.
  • Understand that CNN kernels are learned, not designed by hand.
  • Inspect kernel weights of a Conv2d layer in PyTorch.
Definition

A kernel is the small grid of learnable weights (e.g., 3×3) that a convolution slides across the input. Its numeric values define which local pattern produces a strong response.

Kernel vs. Filter

The words are often used interchangeably, but precisely: a kernel is a 2D weight grid for one channel; a filter is the full stack of kernels across all input channels that produces one output channel. A Conv2d(3, 16, 3) layer has 16 filters, each containing 3 kernels of size 3×3. The next lecture develops this distinction fully.

TermDimensionalityProduces
Kernelk × k (per channel)Response for one input channel
FilterC_in × k × kOne output channel
Conv layer weightsC_out × C_in × k × kAll output channels

Values Decide the Pattern

Before deep learning, engineers designed kernels by hand. These still build intuition for what learned kernels do:

Vertical edge (Sobel) Blur (box) Sharpen -1 0 1 1/9 1/9 1/9 0 -1 0 -2 0 2 1/9 1/9 1/9 -1 5 -1 -1 0 1 1/9 1/9 1/9 0 -1 0

The edge kernel responds strongly where left and right brightness differ; the blur averages neighbors; the sharpen amplifies the center relative to neighbors. A CNN discovers analogous—but far more varied—kernels automatically.

Choosing Kernel Size

1×1

  • Mixes channels, not space.
  • Cheap dimensionality change.
  • Used in bottlenecks.

3×3

  • Modern default.
  • Stacks to large receptive fields.
  • Best accuracy-per-parameter.

5×5 / 7×7

  • Larger receptive field per layer.
  • More parameters and compute.
  • Common only in first layer.

A key insight from VGG: two stacked 3×3 kernels cover the same 5×5 receptive field with fewer parameters and an extra nonlinearity. This is why 3×3 dominates modern CNNs.

Learned, Not Designed

In a CNN the kernel values start random and are updated by backpropagation. Early layers reliably converge toward edge and color-blob detectors; deeper layers form texture and part detectors. You never hand-set them—you inspect them.

import torch from torch import nn conv = nn.Conv2d(in_channels=3, out_channels=16, kernel_size=3) print(conv.weight.shape) # torch.Size([16, 3, 3, 3]) (C_out, C_in, k, k) print(conv.bias.shape) # torch.Size([16]) # Manually set a kernel (rarely done, but instructive) with torch.no_grad(): conv.weight.zero_() conv.weight[0, 0] = torch.tensor([[-1., 0, 1], [-2., 0, 2], [-1., 0, 1]]) # vertical edge
Common Misconception

“Bigger kernels always see more, so they’re better.” A larger kernel costs quadratically more parameters and compute. Stacking small 3×3 kernels usually reaches the same receptive field more cheaply and with more nonlinearity.

Knowledge Check

  1. Short Answer: What is a kernel? Answer: The small grid of learnable weights slid across the input during convolution.
  2. True/False: In CNNs, kernel values are hand-designed. Answer: False—they are learned by backpropagation.
  3. Multiple Choice: A 1×1 kernel primarily: (a) mixes channels, (b) detects large edges, (c) does pooling. Answer: (a).
  4. Short Answer: Difference between a kernel and a filter? Answer: A kernel is one 2D grid per channel; a filter stacks kernels across all input channels to make one output channel.
  5. True/False: Two stacked 3×3 kernels cover a 5×5 receptive field. Answer: True.
  6. Multiple Choice: The modern default kernel size is: (a) 7×7, (b) 3×3, (c) 11×11. Answer: (b).
  7. Short Answer: What does a Sobel kernel detect? Answer: Edges (intensity gradients) in a given direction.
  8. Short Answer: For Conv2d(3, 16, 3), what is weight.shape? Answer: (16, 3, 3, 3).
  9. True/False: Larger kernels cost more parameters and compute. Answer: True.
  10. Multiple Choice: Early-layer learned kernels typically resemble: (a) object detectors, (b) edge/color detectors, (c) random noise after training. Answer: (b).

Key Takeaways

  • A kernel is the sliding weight grid; its values decide what pattern it detects.
  • A filter is the stack of kernels over all input channels producing one output channel.
  • 3×3 kernels dominate: stacking them beats one large kernel in cost and expressiveness.
  • CNN kernels are learned, evolving from edges (early) to object parts (deep).
  • Next, Filters formalizes multi-channel filters and channel counts.
Trainer’s Guide

Hands-on idea: Apply hand-set edge, blur, and sharpen kernels to a grayscale image with F.conv2d and display the results side by side.

Discussion prompt: Why did the field move from hand-designed kernels to learned ones?

Recap: A kernel is a small grid of weights whose values determine the pattern it responds to; CNNs learn these automatically. Continue with Filters.