Translation invariance means keeping a model’s output stable when an image shifts position. A one-pixel shift can otherwise change a CNN’s prediction substantially, even when the image remains semantically identical. section 3.5
Pooling provides partial translation invariance by aggregating nearby values. Max pooling uses a window of size K and commonly moves it by stride K, shrinking each spatial dimension while making small shifts less disruptive. section 3.5 section 3.8
Testing shifted images
Shift one MNIST image one pixel toward the lower right and upper left, then compare softmax predictions. The correct-class probability falls from 0.782 for the original to 0.443 and 0.315 after shifting, exposing weak invariance. section 3.5
class MaxPool2dAA(nn.Module):
def __init__(self, kernel_size=2, ratio=1.7):
"""
kernel_size: how much to pool by
ratio: how much larger the blurring filter should be than the
pooling size
"""
super(MaxPool2dAA, self).__init__()
blur_ks = int(ratio*kernel_size) #1
self.blur = BlurLayer(kernel_size=blur_ks, #2
stride=kernel_size, D=2)
self.kernel_size = kernel_size #3
def forward(self, x):
ks = self.kernel_size
tmp = F.max_pool2d(x, ks, stride=1, padding=ks//2) #4
return self.blur(tmp) #5
#1 Makes a slightly larger filter for blurring
#2 Creates the blur kernel
#3 Stores the pooling size
#4 Applies pooling with stride=1
#5 Blurs the result
Inspect an anti-aliased max-pooling implementation that uses stride-one max pooling followed by a BlurLayer, reducing aliasing and improving prediction consistency across shifts. section 14.1.3
