FP16 — Module 6.3 | Codextroop

How This Lesson Fits the Module

Tensor cores expect 16-bit operands. FP16 (IEEE 754 half precision) packs a float into 16 bits: 1 sign, 5 exponent, 10 mantissa. That halves memory versus FP32 and unlocks tensor core throughput—but the narrow dynamic range makes training trickier than inference.

Module 6.2’s mixed precision lecture applies the training patterns introduced here. FP16 is the classic choice on Volta/Turing consumer GPUs without native BF16.

Learning Objectives

By the end of this lesson, students should be able to:

Describe the FP16 bit layout and compare range/precision to FP32.
Explain why gradients can underflow in FP16 and how loss scaling fixes it.
Train with torch.cuda.amp.autocast and GradScaler.
Identify which ops should stay in FP32 during mixed precision.
Choose FP16 vs FP32 for inference deployment tradeoffs.

FP16 in One Picture

Format	Bits	Exponent	Mantissa	Approx. Range
FP32	32	8 bits	23 bits	≈ 1e-38 to 3e38
FP16	16	5 bits	10 bits	≈ 6e-5 to 65,504

FP16 cannot represent many small gradient values. Values below ~6×10^-8 flush to zero—stalling learning in deep networks without countermeasures.

Memory Impact Storing a 1B-parameter model: ~4 GB in FP32, ~2 GB in FP16. Activations during training scale the same way—directly easing VRAM pressure.

Mixed Precision Training Pattern

Best practice: keep master weights in FP32; run forward/backward matmuls in FP16 inside autocast; use GradScaler to multiply loss before backward, then unscale gradients before the optimizer step.

import torch
from torch.cuda.amp import autocast, GradScaler

device = "cuda"
model = torch.nn.Linear(1024, 1024).to(device)
optimizer = torch.optim.Adam(model.parameters())
scaler = GradScaler()

for x, y in loader:
    x, y = x.to(device), y.to(device)
    optimizer.zero_grad()

    with autocast(dtype=torch.float16):
        logits = model(x)
        loss = torch.nn.functional.cross_entropy(logits, y)

    scaler.scale(loss).backward()
    scaler.step(optimizer)
    scaler.update()

What Stays in FP32?

PyTorch’s autocast maintains a allowlist: loss accumulation, softmax in unstable regimes, batch norm stats, and small reductions often run in FP32 even inside an FP16 forward pass. Trust the policy; force FP32 only when you measure numerical issues.

FP16 Strengths

2× memory savings vs FP32
Tensor core speed on NVIDIA hardware
Mature ecosystem (AMP, ONNX FP16 export)

FP16 Risks

Gradient underflow without scaling
Overflow in large activations
Accumulation error in very deep reductions

Critical Mistake — FP16 Without GradScaler

Casting the entire model to .half() and training without loss scaling often yields NaNs or frozen loss. Use AMP’s autocast + GradScaler, or switch to BF16 on supported hardware.

Misconception — FP16 Always Loses Accuracy

Well-tuned mixed precision training matches FP32 accuracy on most CNNs and transformers. Problems appear in niche ops (large softmax, tiny learning rates)—debug with full FP32 baseline, not assumptions.

FP16 Inference

Inference skips gradients; forward-only FP16 (or INT8 later) cuts latency and VRAM. Export with model.half() and ensure inputs match dtype. Validate accuracy on a golden eval set after conversion.

model.eval()
model.half()
with torch.inference_mode():
    x = torch.randn(1, 3, 224, 224, device="cuda", dtype=torch.float16)
    out = model(x)

Knowledge Check

Short Answer: How many bits in FP16? Answer: 16 (1 sign + 5 exponent + 10 mantissa).
Short Answer: Why use GradScaler? Answer: Multiplies loss so backward gradients stay above FP16 underflow threshold; unscales before optimizer.
True/False: FP16 has wider dynamic range than FP32. Answer: False—FP32 range is much larger.
Multiple Choice: Master weights in mixed precision are usually: (a) FP16, (b) FP32, (c) INT8. Answer: (b).
Short Answer: What does autocast do? Answer: Automatically runs eligible ops in lower precision (FP16) for speed/memory.
Short Answer: Name one op that often stays FP32 in autocast. Answer: Loss accumulation, batch norm, or large softmax reductions.
True/False: Inference requires GradScaler. Answer: False—no backward pass.
Multiple Choice: Primary FP16 training risk: (a) disk full, (b) gradient underflow, (c) overfitting only. Answer: (b).
Short Answer: How much VRAM do FP16 weights save vs FP32? Answer: Roughly half.
Short Answer: Which hardware unit benefits most from FP16 matmul? Answer: Tensor cores.

Key Takeaways

FP16 trades dynamic range for half the memory and tensor core speed.
Train with autocast + GradScaler; don’t blindly cast everything to half.
Keep master weights FP32; let autocast choose op-level precision.
Validate inference after FP16 conversion on real eval data.
Next: BF16 — a 16-bit format with FP32-like range.

Trainer’s Guide

Hands-on idea: Train ResNet-18 on CIFAR-10 in FP32 vs AMP FP16. Compare accuracy, epoch time, and peak VRAM.

Discussion prompt: When would you disable loss scaling while still using autocast?

What’s Next BF16 shares 16 bits but keeps an 8-bit exponent like FP32—often eliminating loss scaling on Ampere and newer GPUs.