A U-Net is the denoising workhorse inside Stable Diffusion. Its U-shaped architecture transforms noisy latent data into a prediction of the noise that should be removed. section 4.3 section 4.3.3
The practical job looks like this:
- Receives noisy latents as part of the image-generation process. section 4.3.3
- Predicts the noise added to those latents rather than directly drawing the finished image. section 4.3.3
- Learns through repeated training examples: predict the noise, compare the result, and adjust its weights. section 4.3.3
- Works alongside the VAE, CLIP encoder, and scheduler, which transform images, transform text, and control noise removal. section 4.3
Swap the image-making personality
A Stable Diffusion pipeline can replace its default U-Net with a custom-trained one. Community checkpoints can target fantasy or photorealistic images, changing the quality and style of generated results without rebuilding the entire pipeline. section 4.4 section 5.0
Figure 4.21 The U-NET is so named because it looks like a “U”

Use the U-shaped diagram to connect the architecture’s name with its role in estimating noise; the diagram keeps the intimidating machinery at a useful distance. section 4.3.3






