Epoch — Module 6.1

How This Lesson Fits the Module

Each batch triggers one optimizer step. An epoch is one complete pass through the entire training dataset (all batches once, with shuffling typically enabled).

Epoch count, learning rate schedules, and early stopping are defined in epochs—the time unit trainers use when reading learning curves.

Learning Objectives

By the end of this lesson, students should be able to:

Define epoch, iteration (step), and their relationship to batch size.
Implement a multi-epoch training loop in PyTorch.
Log per-epoch train and validation metrics.
Apply early stopping when validation loss stops improving.
Estimate total optimizer steps from epochs × batches per epoch.

Epoch vs Step

Definition — Epoch

One epoch means every training example has been used exactly once (in expectation, if sampling with replacement in large-scale systems; exactly once per shuffle in standard DataLoader loops).

One step (iteration) = one batch processed + one optimizer update.

Term	Formula / Meaning
Steps per epoch	⌈N / batch_size⌉
Total steps	epochs × steps per epoch
Examples seen	epochs × N (with replacement in streaming)

Multi-Epoch Training Loop

import torch

def run_epoch(model, loader, optimizer, criterion, train=True):
    model.train(train)
    total_loss, n = 0.0, 0
    ctx = torch.enable_grad() if train else torch.no_grad()
    with ctx:
        for x, y in loader:
            if train:
                optimizer.zero_grad()
            logits = model(x)
            loss = criterion(logits, y)
            if train:
                loss.backward()
                optimizer.step()
            total_loss += loss.item() * x.size(0)
            n += x.size(0)
    return total_loss / n

best_val = float("inf")
patience, bad_epochs = 5, 0

for epoch in range(1, 51):
    train_loss = run_epoch(model, train_loader, optimizer, criterion, True)
    val_loss = run_epoch(model, val_loader, optimizer, criterion, False)
    print(f"epoch {epoch}: train={train_loss:.4f} val={val_loss:.4f}")
    if val_loss < best_val:
        best_val, bad_epochs = val_loss, 0
        torch.save(model.state_dict(), "best.pt")
    else:
        bad_epochs += 1
        if bad_epochs >= patience:
            print("early stop")
            break

Volume 05 Bridge Track validation metrics each epoch like cross-validation folds—stop when generalization stalls (overfitting signal).

How Many Epochs?

There is no universal answer. Small datasets may need 100+ epochs; ImageNet often uses 90–300 with strong augmentation. Watch validation curves—training loss still falling while validation rises means too many epochs without regularization.

Common Misconception: “More epochs always means a better model.”

Reality: Extra epochs after validation peaks waste compute and worsen overfitting unless regularized or early-stopped.

Common Misconception: “Epoch and iteration are interchangeable.”

Reality: One epoch contains many iterations. Learning rate schedulers may step per epoch or per batch—read the API.

Knowledge Check

Short Answer: Define one epoch. Answer: One full pass through the training dataset.
True/False: One epoch always equals one optimizer step. Answer: False—many steps per epoch.
Multiple Choice: N=8000, B=200 → steps/epoch: (a) 40, (b) 200, (c) 8000, (d) 1. Answer: (a).
Short Answer: Purpose of early stopping? Answer: Halt training when validation stops improving to limit overfitting.
True/False: model.eval() should run during validation each epoch. Answer: True.
Multiple Choice: 10 epochs × 100 steps/epoch = total steps: (a) 10, (b) 100, (c) 1000, (d) 10000. Answer: (c).
Short Answer: Why shuffle each epoch? Answer: Reduces order bias and changes batch composition.
True/False: Training loss ↓ and validation loss ↑ suggests overfitting. Answer: True.
Multiple Choice: Next hyperparameter on step size: (a) Learning Rate, (b) Input Layer, (c) Softmax, (d) Bias. Answer: (a).
Short Answer: What to save when val loss improves? Answer: Model checkpoint (state_dict).

Key Takeaways

Epoch = full dataset pass; step = one batch update.
Log train and validation metrics every epoch.
Early stopping saves the best checkpoint, not the last.
Total steps = epochs × ⌈N/B⌉.
Next: Learning Rate — the step size behind every epoch of progress.

Trainer’s Guide

Visualization: Plot train/val loss vs epoch for a small net; students mark the early-stop point.

Discussion: Why do transfer-learning fine-tunes sometimes use only 3–10 epochs?

What’s Next Each epoch applies many small steps controlled by learning rate.