Noise is variation or content that gets in the way of a signal. What counts as noise depends on the goal: a watermark distracts from the subject of a training image, while background sound can make recorded speech harder to hear. In image generation, random noise also has a deliberate role: it is the starting point from which a diffusion model develops an image. section 12.6 section 7.10 section 12.2
From a blurry blob to a tree
Imagine asking a model for a tree. It begins with an unclear field of random noise. Across denoising timesteps, a rough silhouette emerges; later, the trunk and canopy become recognizable; finally, smaller branches and leaf textures come into focus. The aim is to remove what does not fit the intended image, refining broad structure before details. section 12.2
Figure 12.4 An example of denoising
An intermediate image shows why denoising is gradual: some structure is visible, but the intended subject remains ambiguous. section 12.2
To learn this transformation, a U-Net works with a noisy 8×8 image matrix and predicts the noise pattern added to the original clean image. It summarizes the input into more abstract representations, then reconstructs it while removing noise. This training task gives a more concrete meaning to “denoising” than simply sharpening a picture. section 12.16
Noise matters in other stages and tasks, too:
- Before image-model training, remove watermarks, logos, timestamps, and borders so the model does not learn to reproduce those artifacts. section 12.6
- In speech enhancement, removing background noise can help recover speech, but reducing some frequencies may also damage parts of the recording. section 7.10
- In regression, noise can describe variation the included features do not explain: a diamond-price example attributes the remaining 12% of variance to other factors or noise. section 8.12
