Noise — Module 17.1

How This Lesson Fits the Module & Volume

The diffusion overview named three layers. This lecture is layer one: the forward process. Every later idea—denoising, DDPM training, img2img strength, SVD motion noise—is a use of the same Gaussian corruption. If you do not understand the schedule, you cannot debug “too noisy / too clean / burnt CFG” artifacts in Stable Diffusion or SDXL.

Volume 16 treated noise as a slider. Volume 17 treats it as a probability path: β_t, ᾱ_t, and signal-to-noise ratio (SNR). Next lecture flips the arrow.

Learning Objectives

By the end of this lesson, students should be able to:

Explain why diffusion uses isotropic Gaussian noise (closure, CLT, tractable KL).
Define β_t, α_t = 1 − β_t, and ᾱ_t = ∏_s=1..t α_s.
Write and code the closed-form q(x_t | x₀) without simulating every intermediate step.
Compare linear vs cosine β schedules and state what SNR means for training.
Contrast variance-preserving (VP) DDPM noise with variance-exploding (VE) score-SDE noise at a conceptual level.
Connect “noise strength” in img2img / SVD to a starting timestep, not to a different algorithm.

Definition

Noise, in diffusion, is almost always standard Gaussian ε ~ N(0, I) added according to a schedule. In discrete DDPM form, each forward step is q(x_t | x_t−1) = N(x_t; √(1−β_t) x_t−1, β_t I). Because Gaussians compose, the marginal from the clean sample is q(x_t | x₀) = N(x_t; √ᾱ_t x₀, (1−ᾱ_t) I). At t = T, ᾱ_T ≈ 0, so x_T is nearly N(0, I)—the prior you sample from at inference.

Why Gaussian?

You could corrupt images with dropout, blur, or salt-and-pepper. Gaussian noise wins for three engineering reasons: (1) the sum of Gaussians is Gaussian, so many tiny steps collapse to one closed-form draw; (2) KL divergences between Gaussians are analytic, which is how the DDPM variational bound simplifies; (3) high-dimensional isotropic Gaussians concentrate on a thin shell, giving a well-behaved prior. The central limit theorem also says many small independent corruptions look Gaussian anyway—so you might as well model that directly.

The Schedule Alphabet

Symbol	Meaning	Typical range
β_t	Per-step noise variance (DDPM parameterization)	~10⁻⁴ → ~0.02 (linear, T=1000)
α_t	1 − β_t, residual signal scale this step	Just under 1
ᾱ_t	∏ α_s, remaining signal variance from x₀	1 at t=0 → ~0 at t=T
SNR(t)	ᾱ_t / (1 − ᾱ_t)	High (clean) → low (pure noise)
σ_t	Noise scale in some SDE / VE parameterizations	Grows with t (VE)

Linear vs Cosine (and Why It Matters)

Linear β (Ho 2020)

Simple: linspace(1e-4, 0.02).
Destroys high frequencies early.
Still the mental default for DDPM.

Cosine (Nichol & Dhariwal)

Slower drop of ᾱ_t in early t.
More training signal at mid noise.
Common in improved DDPM / some SD stacks.

Later schedules

Zero terminal SNR, offset noise.
Logit-normal t sampling (SD3-class).
Flow models use a different path—see FLUX.

VP vs VE, Briefly

Variance-preserving (VP) processes (classic DDPM) shrink the signal as they add noise so Var(x_t) stays near 1 if x₀ was scaled that way. Variance-exploding (VE) processes (NCSN / some SDEs) leave the signal and pile on increasing σ_t. Same intuition—destroy structure—different algebra. Stable Diffusion’s discrete samplers are VP-family. Score-SDE unification (Song et al.) shows both are discretizations of stochastic differential equations; you do not need the SDE to train SD, but it explains why many samplers exist.

Implementing Schedules and q_sample

Training never loops t = 1..t to build x_t. That would be O(T) per example. Use the closed form. Keep ᾱ on CPU or GPU as a length-T buffer; index it with the batch of timesteps.

import math
import torch

def make_beta_schedule(T=1000, schedule="linear"):
    if schedule == "linear":
        return torch.linspace(1e-4, 0.02, T)
    if schedule == "cosine":
        # Nichol & Dhariwal cosine schedule (s = 0.008)
        s = 0.008
        steps = torch.arange(T + 1, dtype=torch.float64)
        f = torch.cos(((steps / T) + s) / (1 + s) * math.pi / 2) ** 2
        alphas_cumprod = (f / f[0]).clamp(min=1e-4, max=0.9999)
        betas = 1.0 - (alphas_cumprod[1:] / alphas_cumprod[:-1])
        return betas.float().clamp(0.0001, 0.999)
    raise ValueError(schedule)

def snr(alphas_cumprod):
    return alphas_cumprod / (1.0 - alphas_cumprod)

T = 1000
betas = make_beta_schedule(T, "linear")
alphas_cumprod = torch.cumprod(1.0 - betas, dim=0)
print(float(snr(alphas_cumprod)[0]), float(snr(alphas_cumprod)[-1]))
# high SNR near t=0, near-zero SNR at t=T

Noise as a Control, Not a Bug

In img2img you encode a real image (or latent), then partially noise it to some t_start < T and denoise from there. High strength ≈ large t_start ≈ more freedom to ignore the input. Low strength keeps layout. Inpainting noises only the masked region. SVD’s motion bucket / noise aug parameters perturb the conditioning image’s latent so the video can move. Same Gaussian, different mask in time or space.

Get the noise right

Match train and sample schedules.
Scale latents (SD’s 0.18215) before noising.
Fix the RNG seed when you need reproducibility.

Common failure modes

Wrong ᾱ indexing (off-by-one t).
Forgetting VAE scaling → over/under noise.
Assuming “more noise always = worse” (training needs all SNRs).

Common Misconception

“We add a little Gaussian grain like a film filter, then the UNet sharpens it.” Film grain is tiny and optional. Diffusion noise at large t destroys almost all spatial structure—x_T is not a noisy photo, it is a draw from the prior. The network does not “degrain”; it transports a Gaussian toward the data manifold, one SNR step at a time. Also: ε is usually the same dimensionality as x (or z), not a scalar volume knob.

Knowledge Check

Short Answer: What is α_t in terms of β_t? Answer: α_t = 1 − β_t.
True/False: You must simulate all t intermediate Gaussians to obtain x_t from x₀. Answer: False—the closed form q(x_t|x₀) is one draw.
Multiple Choice: SNR(t) = ᾱ_t/(1−ᾱ_t) is typically: (a) increasing in t, (b) decreasing in t, (c) constant. Answer: (b).
Short Answer: Give one reason diffusion uses Gaussian noise. Answer: Gaussians are closed under addition / KLs are analytic / CLT (any one of these).
True/False: A linear β schedule and a cosine schedule produce the same ᾱ_t curve. Answer: False.
Multiple Choice: Classic DDPM noise is: (a) variance-preserving, (b) variance-exploding only, (c) salt-and-pepper. Answer: (a).
Short Answer: In img2img, what does a higher denoise strength correspond to? Answer: Starting the reverse process from a larger timestep t (more corruption of the input).
Short Answer: What is ᾱ_t? Answer: The product of α_s from s=1 to t (remaining signal scale from x₀).
Multiple Choice: At t = T with a well-chosen schedule, x_T is approximately: (a) x₀, (b) N(0, I), (c) the dataset mean. Answer: (b).
True/False: Stable Diffusion noises RGB pixels at 512×512 during UNet training. Answer: False—it noises the VAE latent (after scaling).

Key Takeaways

Forward diffusion is a Gaussian Markov chain with a β schedule; ᾱ_t summarizes how much x₀ remains.
Always use the closed-form q(x_t|x₀) in training; never step noise one t at a time unless you are debugging.
SNR(t) is the right language for “how hard is this timestep.” Linear vs cosine changes where the hard timesteps sit.
Img2img, inpaint, and SVD motion controls are partial applications of the same noise, not new physics.
Next: Denoising—predicting ε and walking back down t.

Trainer’s Guide

Hands-on idea: Plot ᾱ_t and SNR(t) for linear vs cosine on the same axes. Then show the same photo at t corresponding to SNR = 10, 1, 0.1, 0.01 for each schedule—students see that “t=400” is not a universal physical time.

Discussion prompt: If someone trains with cosine ᾱ but samples with a linear scheduler in diffusers, what breaks and why?

Recap: Noise is a scheduled Gaussian path from data to prior, fully determined by ᾱ_t. Continue with Denoising.