U-Net
U-Net is an encoder-decoder neural-network architecture with downsampling and upsampling paths connected by skip connections for combining features at multiple spatial scales.
Build a Text-to-Image Generator (from Scratch)
U-Net is an encoder-decoder neural-network architecture commonly used as the denoising backbone in diffusion models.
- How it is used
- Serves as the noise-predicting backbone for reverse diffusion in image generation, including conditional and latent-space generation.
- How deep it goes
- Traces encoder, bottleneck, decoder, skip connections, training objectives, and attention, progressing from a library model to scratch-built implementations.
- Examples
- Uses Fashion MNIST clothing images, labeled clothing generation, and higher-resolution flower images to demonstrate training and sampling.
- What you can do after
- Build it from scratch as a conditional denoising U-Net, then train and sample with a DDPM; extend the architecture with attention.
Read how Build a Text-to-Image Generator (from Scratch) explains U-Net →
Learn Generative AI with PyTorch
A U-shaped convolutional encoder-decoder neural network with skip connections, used here to predict and remove noise during diffusion-based image generation.
- How it is used
- Predicts noise at each reverse-diffusion step to generate flower images from random noise.
- How deep it goes
- Explains architecture, time embeddings, attention, and training, but presents only part of the provided UNet class.
- Examples
- Uses 64 × 64 Oxford 102 Flower images, a PyTorch UNet configuration, and a 50-step generation process.
- What you can do after
- Operate/configure a PyTorch UNet for noise-prediction training and iterative flower-image generation.
Inside Deep Learning: Math, Algorithms, Models
An encoder-decoder neural network architecture for image segmentation that combines low-level and high-level features via skip connections.
- How it is used
- Deploys encoder-decoder processing for detailed image segmentation and other tasks requiring predictions at multiple image points.
- How deep it goes
- Explains subnetworks, recursive nesting, tensor concatenation, and PyTorch construction, while deferring minute implementation details.
- Examples
- Uses nested UNetBlock2d modules, a one-channel output convolution, validation-loss plots, and comparisons with fine- and coarse-grained models.
- What you can do after
- Build it from scratch in PyTorch, train it for segmentation, and compare its validation loss with alternative architectures.
Read how Inside Deep Learning: Math, Algorithms, Models explains U-Net →
Grokking AI Algorithms, Second Edition
A U-Net is an encoder-decoder neural-network architecture with skip connections commonly used for image segmentation and image generation.
- How it is used
- Deploys an image-to-image CNN architecture for reconstructing images and powering diffusion-based image generation.
- How deep it goes
- Explains the encoder, bottleneck, decoder, skip connections, denoising objective, and reverse-diffusion use without deep mathematical detail.
- Examples
- Uses an 8×8 noisy image, architecture diagrams, and stepwise denoising from pure noise to a generated image.
- What you can do after
- Build it from scratch as the denoising network in a diffusion model and follow its training and generation workflow.
Read how Grokking AI Algorithms, Second Edition explains U-Net →
The AI Pocket Book
An encoder-decoder neural network architecture that reconstructs a full-size output image from a compressed representation, often using intermediate features from the shrinking path.
- How it is used
- Transforms images into full-size reconstructed or stylistically altered images, including caption-guided diffusion outputs.
- How deep it goes
- Explains the shrinking path, abstract representation, intermediate features, and reconstruction, but does not detail layer internals.
- Examples
- Uses damaged-image reconstruction, Van Gogh-style transformation, and caption-guided repair from Gaussian noise.
- What you can do after
- Understand how U-Net-like CNNs reconstruct images and guide iterative text-to-image diffusion.
Related concepts
- convolutional neural networkkind of · A convolutional neural network is a neural-network architecture that applies learned convolutional operations to extract patterns from structured data.
- bottleneck layerhas part · A bottleneck layer is a deliberately narrow intermediate neural-network representation that constrains the dimensionality or information capacity of data passing through the model.
- encoderhas part · An encoder is a component that transforms input data into an encoded representation for subsequent processing, reconstruction, comparison, or transmission.
- neural networkkind of · A neural network is a computational model of interconnected, parameterized units that learns transformations from inputs to outputs.
- contracting encoder pathhas part · A contracting encoder path is the progressively downsampling branch of an encoder-decoder architecture that extracts hierarchical representations.
- expansive decoder pathhas part · An expansive decoder path is the progressively upsampling branch of an encoder-decoder architecture that reconstructs higher-resolution representations.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.