A pointwise convolution uses a 1 × 1 kernel to process the channel stack at each spatial location. It mixes or changes channels without examining neighboring pixels, so it preserves the spatial receptive field. With Cin input channels and Cout output channels, it provides a comparatively cheap way to reshape channel dimensions. section 6.4 section 6.8
Use pointwise convolutions when:
- You need to increase or decrease the number of channels, especially when matching the dimensions of a residual shortcut. section 6.4 section 6.5.3
- You want features at the same location to share information across channels, such as combining several orientation-specific edge detectors into a more general edge feature. section 6.4
- You want a cheaper layer than a 3 × 3 convolution, while recognizing that larger filters can often learn similar channel-sharing behavior. section 6.4 section 6.4.1
A channel-sharing block
In PyTorch, wrap nn.Conv2d(n_filters, n_filters, (1, 1), padding=0) with batch normalization and a LeakyReLU. Insert the block after a couple of ordinary convolutional layers or once per network region. The operation keeps the spatial size while learning a new combination of the existing channels. section 6.4.1
def infoShareBlock(n_filters):
return nn.Sequential(
nn.Conv2d(n_filters, n_filters, (1,1), padding=0),
nn.BatchNorm2d(n_filters),
nn.LeakyReLU())
Inspect the compact PyTorch implementation of a pointwise channel-sharing block. section 6.4.1
Pointwise convolutions become especially important in residual bottlenecks. A short path can use 1 × 1 convolution plus batch normalization to change C to C′, while the long path shrinks channels, applies a normal convolution, and expands them again. This saves parameters and memory, but using pointwise layers everywhere does not guarantee better accuracy. section 6.5.3 section 6.5.4
