PyTorch Unfold extracts sliding local blocks from an input tensor. With an image shaped (N, C, H, W), it returns (N, C × kernel_height × kernel_width, L), where L is the number of blocks. section 4.3.1
Configure extraction through two parameters:
- Set kernel_size to the block dimensions, such as an 8 × 8 image patch. section 4.3.1
- Set stride to the window step, such as 8 for non-overlapping 8 × 8 patches. section 4.3.1
- Transpose and reshape the result when a model needs a sequence of patch vectors. section 4.3.1
Extracting 8 × 8 patches
For one 3-channel image shaped (1, 3, 128, 128), Unfold with kernel_size=8 and stride=8 produces (1, 192, 256): 256 patches, each containing 3 × 8 × 8 = 192 values. Reshaping gives (1, 256, 192), a sequence of patch vectors. section 4.3.1
Listing 4.2 Converting images to patches
def extract_patches(image_tensor, patch_size=8):
bs, c, h, w = image_tensor.size() #1
unfold = torch.nn.Unfold(kernel_size=patch_size, #2
stride=patch_size) #2
unfolded = unfold(image_tensor) #3
unfolded = unfolded.transpose(1, 2).reshape(bs,
-1, c * patch_size * patch_size) #4
return unfolded
Use the implementation to convert batched image tensors into patch sequences with torch.nn.Unfold. section 4.3.1
