A diffusion model creates an image by starting with random noise and refining it step by step. Think of a screen full of TV static gradually becoming a recognizable picture. Rather than drawing each object onto a blank canvas, the model learns which noise to remove. section 12.1 section 12.2
From static to a tree
At first, the pixels look like a meaningless blur. After several denoising steps, a rough silhouette appears, though it could still be mistaken for something else. Later, a trunk and canopy become clear. Further steps sharpen the branches, bark, and leaves. This progression shows why an early result may have the right general shape but lack detail. section 12.2
The model learns this reversal by practicing on real images. During forward diffusion, a clean training image is combined with random noise at a chosen timestep. The resulting noisy image and the noise used to make it provide a training example. A noise schedule controls how much of the original image remains at each timestep; examples for different timesteps are calculated from the original image, not by repeatedly modifying the previous noisy example. section 12.9
Table 12.1 A subset of values for the noise schedule
|
Timestep (t) |
Beta |
Alpha calculation (1 - beta) |
Alpha |
Alpha-Bar calculation |
Alpha-Bar |
| 1 |
0.00010 |
1 - 0.00010 |
0.99990 |
0.99990 |
0.99990 |
| 2 |
0.00012 |
1 - 0.00012 |
0.99988 |
0.99990 × 0.99988 |
0.99978 |
| 3 |
0.00014 |
1 - 0.00014 |
0.99986 |
0.99990 × 0.99988 × 0.99986 |
0.99964 |
| … |
|||||
| 1000 |
0.02000 |
1 - 0.02000 |
0.98000 |
0.00004 |
|
The noise-schedule table shows beta, the per-step noise setting; alpha, the signal retained at a step; and alpha-bar, the cumulative signal retained by a timestep. section 12.9
A simple training and generation workflow is:
- Prepare images and captions, convert them to numerical representations, and create noisy training examples. section 12.3
- Train a U-Net to predict the noise in a noisy image, using its caption as a guide. Compare predicted noise with the actual noise, then update the model’s weights through backpropagation. section 12.3 section 12.23
- To generate an image, start with fresh random noise and use the trained model to denoise it repeatedly, guided by a text prompt. section 12.23 section 12.3
Training longer is not always better. Stopping too early can leave generated images noisy or dull; training too long can cause the model to memorize its training images. Validation loss checks noise prediction on unseen examples, while Fréchet inception distance (FID) compares generated and real images. Checkpoints preserve promising model weights as these measures improve. section 12.30
