U-Net

As explained in Grokking AI Algorithms, Second Edition

A U-Net is an encoder-decoder neural-network architecture with skip connections commonly used for image segmentation and image generation.

A U-Net is a U-shaped convolutional neural network for image-to-image tasks. It compresses an image into an abstract summary, then reconstructs it while preserving fine spatial details through skip connections. section 12.15

Its main parts divide the work between understanding the image and rebuilding it:

• Encoder or downsample path: shrinks feature maps to capture high-level context, such as recognizing that an image contains a face. section 12.15
• Bridge or bottleneck: processes the most compressed and abstract representation at the bottom of the U-shaped path. section 12.15
• Decoder or upsample path: expands feature maps with transposed convolutions and reconstructs full-resolution output. section 12.15
• Skip connections: pass early, high-resolution features directly to matching decoder layers, combining precise location information with abstract meaning. section 12.15

Denoising an 8×8 image

During diffusion training, the U-Net receives an 8×8 noisy matrix and predicts the exact noise added to the clean image. It first summarizes the image to understand its essence, then reconstructs it while removing the predicted noise. During generation, the trained network starts with pure noise and denoises it step by step until a clean image emerges. section 12.16 section 12.23

Figure 12.27 The U-Net architecture

Use the architecture figure to trace the encoder, bottleneck, decoder, and skip connections across the U-shaped network. section 12.15

How Grokking AI Algorithms, Second Edition covers it

How deep it goes

Explains the encoder, bottleneck, decoder, skip connections, denoising objective, and reverse-diffusion use without deep mathematical detail.

Examples

Uses an 8×8 noisy image, architecture diagrams, and stepwise denoising from pure noise to a generated image.

What it assumes

Assumes a fundamental understanding of CNNs while building the U-Net and diffusion workflow from that foundation.

What you can do after

Build it from scratch as the denoising network in a diffusion model and follow its training and generation workflow.

How it connects

Connects convolutional neural networks, skip connections, image-to-image tasks, and diffusion models.

liveBookmanning.com

Read Grokking AI Algorithms, Second Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Build a Text-to-Image Generator (from Scratch)U-Net is an encoder-decoder neural-network architecture commonly used as the denoising backbone in diffusion models.
• 

  liveBookmanning.com
  Learn Generative AI with PyTorchA U-shaped convolutional encoder-decoder neural network with skip connections, used here to predict and remove noise during diffusion-based image generation.
• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsAn encoder-decoder neural network architecture for image segmentation that combines low-level and high-level features via skip connections.
• 

  liveBookmanning.com
  The AI Pocket BookAn encoder-decoder neural network architecture that reconstructs a full-size output image from a compressed representation, often using intermediate features from the shrinking path.

Compare how 5 books explain U-Net →

Related concepts

• convolutional neural networkkind of · A convolutional neural network is a neural-network architecture that applies learned convolutional operations to extract patterns from structured data.
• bottleneck layerhas part · A bottleneck layer is a deliberately narrow intermediate neural-network representation that constrains the dimensionality or information capacity of data passing through the model.
• encoderhas part · An encoder is a component that transforms input data into an encoded representation for subsequent processing, reconstruction, comparison, or transmission.
• neural networkkind of · A neural network is a computational model of interconnected, parameterized units that learns transformations from inputs to outputs.
• contracting encoder pathhas part · A contracting encoder path is the progressively downsampling branch of an encoder-decoder architecture that extracts hierarchical representations.
• expansive decoder pathhas part · An expansive decoder path is the progressively upsampling branch of an encoder-decoder architecture that reconstructs higher-resolution representations.

This explainer was generated by Manning's knowledge graph from the text of Grokking AI Algorithms, Second Edition. The section links point to the parts of the book it draws on.