liveBook · Manning

PyTorch Unfold

As explained in Build a Text-to-Image Generator (from Scratch)

PyTorch Unfold is a tensor operation that extracts sliding local blocks from an input tensor.

PyTorch Unfold extracts sliding local blocks from an input tensor. With an image shaped (N, C, H, W), it returns (N, C × kernel_height × kernel_width, L), where L is the number of blocks. section 4.3.1

Configure extraction through two parameters:

  • Set kernel_size to the block dimensions, such as an 8 × 8 image patch. section 4.3.1
  • Set stride to the window step, such as 8 for non-overlapping 8 × 8 patches. section 4.3.1
  • Transpose and reshape the result when a model needs a sequence of patch vectors. section 4.3.1

Extracting 8 × 8 patches

For one 3-channel image shaped (1, 3, 128, 128), Unfold with kernel_size=8 and stride=8 produces (1, 192, 256): 256 patches, each containing 3 × 8 × 8 = 192 values. Reshaping gives (1, 256, 192), a sequence of patch vectors. section 4.3.1

Listing 4.2 Converting images to patches
def extract_patches(image_tensor, patch_size=8):
    bs, c, h, w = image_tensor.size()  #1
    unfold = torch.nn.Unfold(kernel_size=patch_size,  #2
                             stride=patch_size)   #2
    unfolded = unfold(image_tensor)  #3
    unfolded = unfolded.transpose(1, 2).reshape(bs,
                    -1, c * patch_size * patch_size)  #4
    return unfolded

Use the implementation to convert batched image tensors into patch sequences with torch.nn.Unfold. section 4.3.1

How Build a Text-to-Image Generator (from Scratch) covers it

How deep it goes
Explains parameters, tensor shapes, arithmetic, and reshaping, but stops short of Unfold internals.
Examples
Uses 128 × 128 RGB images, 8 × 8 patches, 256 blocks, and 192-value patch vectors.
What it assumes
Assumes familiarity with batched image tensor shapes and teaches patch dimensions through explicit arithmetic.
What you can do after
Apply it via PyTorch to produce batched image-patch tensors for downstream encoding.
How it connects
Connects sliding local blocks with image patches, tensor reshaping, and vision transformers.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build a Text-to-Image Generator (from Scratch). The section links point to the parts of the book it draws on.