A diffusion model generates images by learning to undo noise. During training, clean data is gradually perturbed until its structure disappears. A neural network learns to predict the added noise; during generation, repeated denoising turns a random starting point into a coherent result. section 10.10 section 10.8
The basic workflow has three parts:
- Add noise in steps. A noise schedule sets a beta value for each step, controlling how the original data gives way to noise. section 10.5
- Train a network to predict the noise added at a chosen timestep. Compare its prediction with the actual noise using mean squared error. section 10.8
- Generate by repeatedly using the predicted noise to move from random data toward a recognizable sample. section 10.10
A shape made from points
For a small-scale example, represent a logo outline as a tensor of (x, y) points. Noise gradually scatters the points until the outline is unrecognizable. A trained feedforward network predicts that noise so repeated reverse steps can bring random points toward the original shape. The same principles can be applied to image datasets. section 10.4 section 10.8
def diffuse_points(points, beta):
new_mean = torch.sqrt(1-beta) * points
noise = torch.randn(points.shape)
perturbation = torch.sqrt(beta) * noise
points = new_mean + perturbation
return points, noise
This PyTorch function shows one forward step: scale the existing points, generate random noise, and add a beta-scaled perturbation. section 10.5
For image editing, a text prompt describes the desired result and an input image provides a starting layout. An optional mask confines changes to its selected region: inpainting preserves the unmasked content while denoising repaints the rest. Without a mask, the whole image can be re-rendered. section 2.2.1
