liveBook · Manning

U-Net

Also called UNet

U-Net is an encoder-decoder neural-network architecture with downsampling and upsampling paths connected by skip connections for combining features at multiple spatial scales.

How 7 Manning books explain it

Build a Text-to-Image Generator (from Scratch)

U-Net is an encoder-decoder neural-network architecture commonly used as the denoising backbone in diffusion models.

How it is used
Serves as the noise-predicting backbone for reverse diffusion in image generation, including conditional and latent-space generation.
How deep it goes
Traces encoder, bottleneck, decoder, skip connections, training objectives, and attention, progressing from a library model to scratch-built implementations.
Examples
Uses Fashion MNIST clothing images, labeled clothing generation, and higher-resolution flower images to demonstrate training and sampling.
What you can do after
Build it from scratch as a conditional denoising U-Net, then train and sample with a DDPM; extend the architecture with attention.

Read how Build a Text-to-Image Generator (from Scratch) explains U-Net →

Learn Generative AI with PyTorch

A U-shaped convolutional encoder-decoder neural network with skip connections, used here to predict and remove noise during diffusion-based image generation.

How it is used
Uses U-Net as a noise predictor in diffusion-based flower image generation.
How deep it goes
Explains encoder, decoder, skip connections, attention, and time embeddings, then walks through model configuration and training.
Examples
Uses 64 × 64 Oxford 102 Flower images, a 133-million-parameter U-Net, and a 50-step generation run.
What you can do after
Builds and trains a denoising U-Net with PyTorch, then uses it for iterative flower image generation.

Read how Learn Generative AI with PyTorch explains U-Net →

Inside Deep Learning: Math, Algorithms, Models

An encoder-decoder neural network architecture for image segmentation that combines low-level and high-level features via skip connections.

How it is used
Uses multiscale features to make a prediction for every pixel in an image.
How deep it goes
Explains the three subnetworks and implements a reusable PyTorch block that can be nested into a larger architecture.
Examples
Implements nested UNetBlock2d modules and compares segmentation results with fine- and coarse-grained models.
What you can do after
Builds a segmentation network by nesting UNetBlock2d modules and adding a final convolution for the desired output channels.

Read how Inside Deep Learning: Math, Algorithms, Models explains U-Net →

Local Image Generation

A U-Net is an encoder-decoder neural-network architecture with skip connections for transforming spatial data.

How it is used
Uses a neural network to estimate noise during Stable Diffusion image generation.
How deep it goes
Explains the architecture’s purpose, learned weights, and training loop, but defers detailed network mechanics.
Examples
Uses a U-shaped architecture diagram, Stable Diffusion pipeline, and custom checkpoints for fantasy or photorealistic images.
What you can do after
Apply it via Hugging Face Diffusers by loading a pipeline and replacing its U-Net with a task-specific checkpoint.

Read how Local Image Generation explains U-Net →

Grokking AI Algorithms, Second Edition

A U-Net is an encoder-decoder neural-network architecture with skip connections commonly used for image segmentation and image generation.

How it is used
Deploys an image-to-image CNN architecture for reconstructing images and powering diffusion-based image generation.
How deep it goes
Explains the encoder, bottleneck, decoder, skip connections, denoising objective, and reverse-diffusion use without deep mathematical detail.
Examples
Uses an 8×8 noisy image, architecture diagrams, and stepwise denoising from pure noise to a generated image.
What you can do after
Build it from scratch as the denoising network in a diffusion model and follow its training and generation workflow.

Read how Grokking AI Algorithms, Second Edition explains U-Net →

Build AI Drug Discovery Pipelines

U-Net is an encoder-decoder neural-network architecture with skip connections that combines representations at multiple spatial scales.

How it is used
Uses multiscale sequence features to predict protein residue contacts for constructing graph representations.
How deep it goes
Explains encoder-decoder flow, skip connections, feature inputs, and thresholded outputs, but defers layer-level implementation.
Examples
Uses protein contact prediction, 72 MSA-derived features, contact probabilities, and a 0.5 graph-construction threshold.
What you can do after
Apply U-Net contact predictions to construct protein graphs whose nodes are residues and edges represent predicted contacts.

Read how Build AI Drug Discovery Pipelines explains U-Net →

The AI Pocket Book

An encoder-decoder neural network architecture that reconstructs a full-size output image from a compressed representation, often using intermediate features from the shrinking path.

How it is used
Transforms images into full-size reconstructed or stylistically altered images, including caption-guided diffusion outputs.
How deep it goes
Explains the shrinking path, abstract representation, intermediate features, and reconstruction, but does not detail layer internals.
Examples
Uses damaged-image reconstruction, Van Gogh-style transformation, and caption-guided repair from Gaussian noise.
What you can do after
Understand how U-Net-like CNNs reconstruct images and guide iterative text-to-image diffusion.

Read how The AI Pocket Book explains U-Net →

Also covered in

What to know first

Related concepts

Questions about U-Net

What is U-Net?

U-Net is an encoder-decoder neural-network architecture with downsampling and upsampling paths connected by skip connections for combining features at multiple spatial scales.

Which Manning books explain U-Net?

Build a Text-to-Image Generator (from Scratch), Learn Generative AI with PyTorch, Inside Deep Learning: Math, Algorithms, Models, Local Image Generation, Grokking AI Algorithms, Second Edition, Build AI Drug Discovery Pipelines and The AI Pocket Book.

What should I know before learning U-Net?

Manning books that teach U-Net usually introduce convolutional layer, convolutional neural network, parameter and diffusion model first.

What is U-Net a kind of?

U-Net is a kind of convolutional neural network. A convolutional neural network is a neural-network architecture that processes structured inputs with convolutional operations, typically using learned local filters to capture patterns.

What are the parts of U-Net?

U-Net includes bottleneck layer, contracting encoder path, encoder and expansive decoder path.

What is U-Net also called?

U-Net is also called UNet.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.