Batch — Module 6.1

How This Lesson Fits the Module

SGD and Adam update weights after each batch—a contiguous group of training examples processed together in one forward-backward pass.

Understanding batches connects data loading, GPU utilization, and gradient noise. Batch size is the knob you turn next.

Learning Objectives

By the end of this lesson, students should be able to:

Define a training batch and its tensor shape conventions.
Build batches with DataLoader and TensorDataset.
Explain why batch dimension leads tensor shapes (NCHW for images).
Distinguish training batch from inference batch and “batch” in BatchNorm.
Count optimizer steps per epoch from dataset size and batch size.

What Is a Batch?

A batch is a subset of training examples presented to the network simultaneously. The leading dimension of input tensors is the batch dimension B.

Definition — Mini-Batch

A mini-batch is a batch smaller than the full training set. Gradient is averaged (or summed) over the batch before optimizer step. This is standard GPU training practice.

Tensor	Typical Shape	Meaning
Image batch	(B, C, H, W)	B images, C channels
Tabular batch	(B, F)	B rows, F features
Labels	(B,)	One class index per example
Text batch	(B, T)	B sequences of length T (padded)

DataLoader in PyTorch

from torch.utils.data import DataLoader, TensorDataset
import torch

X = torch.randn(1000, 20)
y = torch.randint(0, 2, (1000,))
dataset = TensorDataset(X, y)

train_loader = DataLoader(
    dataset,
    batch_size=64,
    shuffle=True,       # new random order each epoch
    num_workers=2,      # parallel loading (0 on Windows notebooks OK)
    pin_memory=True,    # faster CPU→GPU copy when using CUDA
    drop_last=False,    # keep partial final batch
)

for x_batch, y_batch in train_loader:
  # x_batch: (64, 20) except possibly last batch
    pass

Volume 05 Bridge Batches come from the training set only—never shuffle validation/test into training batches.

Terminology Trap: Three “Batches”

Training Batch

DataLoader output per iteration
Drives one gradient estimate
shuffle=True common

Batch Normalization

Normalizes activations across batch dim
Different concept—see Batch Normalization
batch size affects BN statistics

Common Misconception: “Larger batch always trains faster.”

Reality: Larger batches improve GPU throughput but may need fewer updates per epoch and different learning rates. Throughput ≠ better validation accuracy.

Common Misconception: “The last batch must always be full size.”

Reality: The final batch may be smaller unless drop_last=True. BatchNorm-heavy models sometimes set drop_last=True for stable statistics.

Knowledge Check

Short Answer: Which tensor dimension is usually the batch? Answer: The first dimension (index 0).
True/False: shuffle=True reorders data each epoch. Answer: True.
Multiple Choice: MNIST batch shape with B=32: (a) (32,28,28), (b) (32,1,28,28), (c) (1,32,28,28), (d) (784,32). Answer: (b) with channels.
Short Answer: What does DataLoader return each iteration? Answer: One batch of inputs and matching labels.
True/False: pin_memory helps GPU training from CPU tensors. Answer: True (with CUDA).
Multiple Choice: drop_last=True discards: (a) first batch, (b) incomplete final batch, (c) labels, (d) gradients. Answer: (b).
Short Answer: Steps per epoch formula with N samples, batch B? Answer: ceil(N / B) (or floor if drop_last).
True/False: Training batch and BatchNorm batch mean the same thing. Answer: False.
Multiple Choice: Next lecture tuning batch count: (a) Batch Size, (b) Sigmoid, (c) Hidden Layer, (d) Loss Functions. Answer: (a).
Short Answer: Why vectorize batches on GPU? Answer: Parallel ops across examples—much faster than one-by-one.

Key Takeaways

A batch is a group of examples sharing one forward-backward-update cycle.
DataLoader handles batching, shuffling, and parallel loading.
Leading tensor dimension is batch size B.
“Batch” in BatchNorm is a related but distinct idea.
Next: Batch Size — tradeoffs when choosing B.

Trainer’s Guide

Exercise: Given N=10,000 and batch_size=128, compute steps per epoch with and without drop_last.

Debug scenario: Model expects (B,3,224,224) but loader yields (B,224,224,3)—students fix channel order.

What’s Next Batch size shapes training dynamics — see Batch Size.