Denoising removes predicted noise from a noisy image in small steps. In a diffusion model, it is less like painting a picture stroke by stroke and more like bringing a shape into focus: the model repeatedly decides what noise to take away. section 12.9 section 12.2
From a noisy blob to a tree
Start with random noise. After several denoising steps, a vague silhouette appears. Further steps make a trunk and canopy recognizable; later ones refine branches and texture. The first steps establish broad structure, while the final steps make smaller adjustments. section 12.2 section 12.25
Training and generation use noise in opposite directions:
- During training, forward diffusion makes noisy examples from clean images. The saved noise patterns let a U-Net practice predicting the noise at a given timestep; prediction errors are used to update its weights. section 12.9 section 12.3
- During generation, start with random noise. At each timestep, the U-Net uses the current image, timestep, and text-prompt embedding to predict noise. Subtract an amount set by the noise schedule, then repeat until an image emerges. section 12.25 section 12.3
Figure 12.54 The denoised images from timestep 1000 to timestep 1
Follow the images from timestep 1000 to timestep 1 to see repeated noise removal turn a noisy starting point into a clearer result. section 12.25
The noise schedule links the two directions. Its alpha and beta values govern how noise is added during training and how much predicted noise is removed during generation. Larger early corrections give way to smaller, cautious refinements. A schedule that adds noise too harshly can make denoising difficult to learn; one that is too gentle can make later steps costly. section 12.25 section 12.28
Denoising can also edit an existing image. For inpainting, a mask marks the region to replace; after each prediction, known pixels outside the mask are restored so the surrounding image stays in place. Outpainting uses a blank area beyond the image border as the region to fill. section 12.31
