Image Outpainting — Module 17.1

How This Lesson Fits the Module & Volume

Inpainting fills a hole inside the frame. Outpainting (uncrop / zoom-out) grows the canvas: pad the image, mark the new border as the mask, and run the same inpaint denoise so the model invents plausible surroundings. It is not a new physics chapter—it is mask geometry plus seam strategy on top of latents and denoising.

Bases remain SD / SDXL / FLUX. ControlNet depth/canny on the padded canvas can lock horizon lines. Production UIs: ComfyUI (Pad + Inpaint nodes) and A1111 (Poor man’s outpainting / dedicated scripts).

Learning Objectives

By the end of this lesson, students should be able to:

Define outpainting as inpainting the new region after canvas expansion.
Build a padded image + border mask and call an SD inpaint pipeline.
Plan overlap, multi-side passes, and lower-strength blend passes to hide seams.
Know when ControlNet depth/canny on the padded frame helps geometry.
Contrast outpaint with txt2img-at-wider-size (no source identity) and simple resize/crop.
Wire the idea in ComfyUI / A1111 without treating outpaint as magic.

Definition

Image outpainting extends an existing picture beyond its original bounds. In latent diffusion practice you (1) enlarge the canvas (pixel pad or latent pad), (2) copy the original into the center (or to one side), (3) set the mask to the unknown border, (4) run an inpaint model so denoising invents the missing context while the original rectangle is conditioned as keep. Overlap between keep and fill is the seam budget.

Inpaint Geometry, Larger Frame

1. Choose expand

Left/right/all sides, target aspect

2. Pad canvas

Fill unknown with gray/edge/noise

3. Mask border

White = new, black = original (+overlap)

4. Inpaint denoise

Prompt the scene, not only the old crop

5. Optional refine

Second pass, lower strength

Strategy	How	When
One-shot all sides	Pad once, one inpaint	Modest expand, simple backgrounds
Side-by-side passes	Outpaint right, then left, then top…	Large expand; less drift per step
Overlap band	Mask includes a strip of original	Always—gives the model a blend zone
ControlNet assist	Depth/canny of padded preview	Architecture, horizon, furniture rows

Outpaint via the inpaint pipeline

There is usually no separate “OutpaintPipeline” in diffusers—you reuse StableDiffusionInpaintPipeline (or XL/FLUX fill). Prompt the full intended scene (“wide coastal meadow, same lighting”), not only the original caption. Keep resolution on-grid for the checkpoint (multiples of 8/64).

from diffusers import StableDiffusionInpaintPipeline
from PIL import Image, ImageOps
import torch

pipe = StableDiffusionInpaintPipeline.from_pretrained(
    "runwayml/stable-diffusion-inpainting",
    torch_dtype=torch.float16,
).to("cuda")

src = Image.open("portrait_square.png").convert("RGB").resize((512, 512))
pad = 128  # expand each side; total 768×768
overlap = 24  # regenerate a strip of the original so seams blend
canvas = ImageOps.expand(src, border=pad, fill=(127, 127, 127))

mask = Image.new("L", canvas.size, 255)  # white = inpaint (new border)
# black = keep the core of the original (inset by overlap)
keep_box = (
    pad + overlap,
    pad + overlap,
    pad + src.width - overlap,
    pad + src.height - overlap,
)
mask.paste(0, keep_box)

wide = pipe(
    prompt="same woman in a linen coat, wide windswept beach, matching daylight, no text",
    image=canvas,
    mask_image=mask,
    num_inference_steps=32,
    strength=0.88,
    guidance_scale=7.5,
).images[0]
wide.save("portrait_outpaint.png")

# Production: multiple directional passes; optional ControlNet depth on canvas

Outpaint vs Just Generating Wider

Outpaint

Keeps the original crop identity
Needs mask + inpaint ckpt
Seams are the hard part

txt2img at new aspect

No pixel loyalty to the source
Cleaner if identity can change
Wrong tool for “extend this still”

Naive resize/crop

No new content
Upscale ≠ invent left side
Use ESRGAN after outpaint if needed

Strengths and Tradeoffs

Strengths

Aspect-ratio rescue for ads and film stills
Same mental model as inpaint
Works in A1111 / ComfyUI graphs

Tradeoffs

Style drift on huge expands
Repeated objects / broken perspective
VRAM jumps with canvas size

Related Lectures

Lecture	Why it sits beside outpainting
Inpainting	Same denoise; different mask geometry
ControlNet	Depth/canny to keep horizon after pad
SDXL / FLUX	Higher-res expand bases
ComfyUI	PadImage + Inpaint node graphs
Automatic1111	UI scripts for poor-man’s outpaint

Common Misconception

“Outpainting is a different model family from inpainting.” In this stack it is almost always the inpaint UNet plus padding. Second: expanding 512→2048 in one shot with no overlap—expect a collage of new rooms. Third: prompting only the original subject name with no scene words; the border has nothing to latch onto.

Knowledge Check

Short Answer: How do you turn outpaint into an inpaint job? Answer: Pad the canvas and mask the new (unknown) border, then run an inpaint pipeline.
True/False: diffusers usually ships a totally separate OutpaintPipeline you must use. Answer: False—reuse inpaint + pad/mask.
Multiple Choice: Overlap on the keep region is for: (a) seam blending, (b) BPE merges, (c) Redis pub/sub. Answer: (a).
Short Answer: Why outpaint one side at a time on huge expands? Answer: Less geometric/style drift per step than all sides at once.
True/False: txt2img at a wider size preserves the original pixels. Answer: False—it does not keep source identity.
Multiple Choice: Helpful conditioner for horizons after pad: (a) ControlNet depth/canny, (b) Naive Bayes, (c) MCP stdio. Answer: (a).
Short Answer: What should the prompt describe? Answer: The full intended scene (including new surroundings), matching lighting.
True/False: Upscaling alone invents a new left side of the photo. Answer: False—upscale adds pixels, not new content beyond the crop.
Multiple Choice: Next production UI lecture: (a) ComfyUI, (b) CBOW, (c) Vol. 05 Ridge. Answer: (a).
Short Answer: Name one VRAM issue with outpaint. Answer: Larger canvas → larger latents / more memory than the original crop.

Key Takeaways

Outpaint = pad + mask the border + inpaint denoise.
Overlap and multi-pass expand beat one giant leap.
Prompt the whole scene; optional ControlNet for geometry.
Not the same as txt2img-wider or naive upscale.
Continue with ComfyUI.

Trainer’s Guide

Lab: Square portrait → 16:9 outpaint in one shot vs left-then-right passes. Compare horizon continuity. Optional ControlNet depth on the padded preview.

Whiteboard: Draw original rect, pad, overlap band, mask. Label “same inpaint UNet.” Arrow to ComfyUI Pad + Inpaint nodes.

Recap: Outpainting extends the frame by inpainting the unknown border. Continue with ComfyUI.