DDIM — Module 17.1

How This Lesson Fits the Module & Volume

DDPM gave you a trainable ε_θ and a slow stochastic reverse. DDIM (Song, Meng, Ermon, ICLR 2021, Denoising Diffusion Implicit Models) is the first sampler every SD practitioner actually uses: same weights, non-Markovian forward process with identical marginals q(x_t|x₀), and a reverse that can skip timesteps. Vol. 16 UI “DDIM / 20 steps / seed lock” is this paper. Stable Diffusion and SDXL still ship DDIM as a first-class scheduler in diffusers.

Later ODE solvers (DPM-Solver, Euler ancestral, UniPC) generalize the same idea. Learn DDIM’s η and inversion once; the rest of the sampler zoo is variations.

Learning Objectives

By the end of this lesson, students should be able to:

Explain why DDIM needs no retraining: it preserves DDPM’s q(x_t|x₀) marginals.
Contrast Markovian DDPM reverse with DDIM’s implicit non-Markovian process.
Use η to interpolate stochasticity: η=0 deterministic, η=1 DDPM-like.
Subsample a timestep trajectory (e.g. 1000 → 50) and implement one DDIM update.
Describe DDIM inversion: encode a real image by running the deterministic reverse backwards.
Know when to pick DDIM vs ancestral DDPM vs faster multistep ODE solvers in production.

Definition

DDIM defines a family of inference processes that share DDPM’s training marginals q(x_t | x₀) but are not required to be Markovian. Given ε_θ, each step predicts x̂₀, then constructs x_t−1 as a mixture of that x̂₀, a direction pointing back toward x_t, and an optional Gaussian whose scale is η. When η = 0 the path is deterministic: one x_T (plus condition) maps to one x₀. Because the generative ODE can be discretized with far fewer than T steps, DDIM made diffusion interactive.

Same Training, Different Reverse

	DDPM	DDIM
Forward used at train time	Markov q(x_t\|x_t−1)	Same L_simple / same ε_θ
Marginal q(x_t\|x₀)	√ᾱ_t x₀ + √(1−ᾱ_t) ε	Identical by construction
Reverse dependence	Only on x_t (Markov)	May depend on x_t and implied x₀ (non-Markovian)
Stochasticity	Always σ_t z (t>0)	η ∈ [0,1] knob
Typical eval steps	~T (1000)	20–100 subsequence
Seed lock / inversion	Weak (extra z)	Strong at η=0

The Update Rule

From ε_θ(x_t, t) form x̂₀ = (x_t − √(1−ᾱ_t) ε_θ) / √ᾱ_t. Pick the next (possibly skipped) index t' < t. DDIM sets

x_t' = √ᾱ_t' x̂₀ + √(1−ᾱ_t' − σ_t²) · ε_θ + σ_t z

with σ_t = η · √[(1−ᾱ_t')/(1−ᾱ_t)] · √(1 − ᾱ_t/ᾱ_t'). η = 0 ⇒ σ_t = 0 ⇒ fully deterministic. η = 1 recovers a DDPM-like variance. Skipping means t' is not t−1: you choose a rising subsequence of ᾱ levels (uniform in t, or spaced in log-SNR).

η = 0

Reproducible stills from a seed.
Invertible (approximately) for edits.
Default mental model for img2img walks.

0 < η < 1

Some diversity from one x_T.
Can hide tiny artifacts.
Harder to invert cleanly.

η = 1

Closest to ancestral DDPM.
Use when you want stochastic texture.
Not what people mean by “DDIM” in UIs.

Implementing a Deterministic DDIM Step

import torch
from diffusers import DDIMScheduler, StableDiffusionPipeline

pipe = StableDiffusionPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16
).to("cuda")
pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config)
pipe.scheduler.set_timesteps(30, device="cuda")   # 30 DDIM steps, not 1000

image = pipe(
    "a brass telescope on a wooden desk, window light, no text",
    num_inference_steps=30,
    guidance_scale=7.5,
    generator=torch.Generator("cuda").manual_seed(42),
).images[0]
image.save("ddim30_seed42.png")

# Manual eta: diffusers DDIMScheduler.step(..., eta=0.0) is deterministic.

A from-scratch step (pixel or latent—same algebra) for teaching:

def ddim_step(x_t, eps_pred, a_bar_t, a_bar_prev, eta=0.0):
    x0_hat = (x_t - (1.0 - a_bar_t).sqrt() * eps_pred) / a_bar_t.sqrt()
    sigma = eta * ((1 - a_bar_prev) / (1 - a_bar_t) * (1 - a_bar_t / a_bar_prev)).sqrt()
    dir_xt = (1.0 - a_bar_prev - sigma**2).clamp(min=0).sqrt() * eps_pred
    noise = sigma * torch.randn_like(x_t) if eta > 0 else 0.0
    return a_bar_prev.sqrt() * x0_hat + dir_xt + noise

Inversion and Editing

Because η = 0 is an ODE discretization, you can run it forward in noise time on a real (latent) image: predict ε, step toward higher t, obtain an x_T that approximately reconstructs x₀ when you denoise again. That is DDIM inversion, the backbone of many “edit this photo but keep layout” tricks, prompt-to-prompt style attention edits, and some style-transfer graphs in ComfyUI. Approximate is the honest word: CFG, large skips, and VAE round-trips all leak. For production identity lock you will still want ControlNet or IP-Adapter—inversion is a sampler tool, not a product guarantee.

Pick DDIM when

You need seed-stable stills or inversion.
20–50 steps on an SD 1.5/XL UNet.
You are teaching sampler math (one clean formula).

Move on when

You want max quality per step → DPM++ / UniPC.
You want ancestral texture → Euler a / DDPM.
Distilled XL Turbo / FLUX schnell already use few-step recipes.

Common Misconception

“DDIM is a different trained model, and fewer steps always mean worse images.” DDIM is almost always the same checkpoint as DDPM training. Quality vs steps is U-shaped: too few under-integrates the ODE; too many with a mismatched scheduler can oversmooth. Also, η=0 does not make CFG deterministic by itself if your pipeline still injects extra noise (img2img strength, SVD noise aug, or a stochastic scheduler wrapper).

Knowledge Check

Short Answer: Do you retrain ε_θ to switch from DDPM sampling to DDIM? Answer: No—DDIM reuses the DDPM-trained denoiser.
True/False: DDIM’s q(x_t|x₀) differs from DDPM’s. Answer: False—marginals match by design.
Multiple Choice: η = 0 means: (a) maximum ancestral noise, (b) deterministic DDIM, (c) CFG off. Answer: (b).
Short Answer: What does DDIM predict first inside a step, before building x_t'? Answer: x̂₀ (from ε_θ and current x_t).
True/False: DDIM reverse is required to be a Markov chain on single-step neighbors t → t−1. Answer: False—it is non-Markovian and may skip to t'.
Multiple Choice: DDIM inversion is most reliable when: (a) η=1 and CFG=15, (b) η=0 and moderate skips, (c) the VAE is skipped entirely. Answer: (b).
Short Answer: Why can DDIM use 50 steps instead of 1000? Answer: It discretizes an implicit ODE / non-Markovian process that shares marginals, so you may jump in t.
Short Answer: What does η = 1 approximate? Answer: DDPM-like stochastic reverse variance.
Multiple Choice: Seed-locked A1111/Comfy stills usually assume: (a) η=0 DDIM-class, (b) extra z every step, (c) GAN latent walk. Answer: (a).
True/False: DPM-Solver and DDIM train two different UNets. Answer: False—they are different samplers on (typically) the same ε_θ.

Key Takeaways

DDIM keeps DDPM’s training marginals and ε_θ; it changes the reverse process.
η=0 is deterministic and invertible-ish; η=1 is stochastic/DDPM-like.
Subsampling timesteps is how 20–50 step UIs exist.
Inversion + prompt edit is a sampler technique; ControlNet still wins hard layout lock.
Next: Stable Diffusion—VAE + CLIP + UNet + CFG assembled as a productized LDM.

Trainer’s Guide

Hands-on idea: Same SD 1.5 checkpoint, same seed, DDIM at 8 / 20 / 50 / 100 steps. Then repeat with η=0 vs η=1 at 30 steps. Students should leave believing “sampler ≠ checkpoint.”

Discussion prompt: If DDIM inversion reconstructs a photo at η=0, why does changing the prompt mid-way sometimes destroy identity anyway? (CFG, skip size, attention leakage.)

Recap: DDIM is the fast, optionally deterministic sampler on a DDPM-trained net. Continue with Stable Diffusion.