concept padding in category deep learning

This is an excerpt from Manning's book Deep Learning with JavaScript: Neural networks in TensorFlow.js. Login to get full access to this book.

Recall from chapter 4 that the output of a convolutional layer has the NHWC shape [numExamples, height, width, channels]. Here, we are dealing with a single input image, so numExamples is 1. We want to visualize the output of each convolutional layer along three remaining dimensions: height, width, and channels. The height and width of a convolutional layer’s output are determined by its filter size, padding, and strides, as well as the height and width of the layer’s input. In general, they get smaller and smaller as you go deeper into a convnet. On the other hand, the value of channels generally gets larger as you go deeper, as the convnet extracts a larger and larger number of features through successive layers of representation transformation. These channels of convolutional layers cannot be interpreted as different color components. Instead, they are the learned feature dimensions. This is why our visualization breaks them into separate panels and draws them in grayscale. Figure 7.8 shows the activations from five convolutional layers of VGG16 given the cat.jpg input image.

Sequence truncation and padding

Now that we have conv1d in our arsenal for text-oriented machine learning, are we ready to train a 1D convnet on the IMDb data? Not quite yet. There is one more thing to explain: truncating and padding of sequences. Why do we need to do truncation and padding? TensorFlow.js models require the inputs to fit() to be a tensor, and a tensor must have a concrete shape. Therefore, although our movie reviews don’t have a fixed length (recall that they vary between 10 and 2,400 words), we have to pick a specific length as the second dimension of the input feature tensor (maxLen), so that the full shape of the input tensor is [numExamples, maxLen]. No such problem existed when we used multi-hot encoding in the previous section because tensors from multi-hot encoding had a second tensor dimension unaffected by sequence length.

This is an excerpt from Manning's book Deep Learning with R. Login to get full access to this book.

If you want to get an output feature map with the same spatial dimensions as the input, you can use padding. Padding consists of adding an appropriate number of rows and columns on each side of the input feature map to make it possible to fit center convolution windows around every input tile. For a 3 × 3 window, you add one column on the right, one column on the left, one row at the top, and one row at the bottom. For a 5 × 5 window, you add two rows (see figure 5.6).

Figure 5.6. Padding a 5 × 5 input in order to extract 25 3 × 3 patches

In layer_conv_2d layers, padding is configurable via the padding argument, which takes two values: "valid", which means no padding (only valid window locations will be used); and "same", which means “pad in such a way as to have an output with the same width and height as the input.” The padding argument defaults to "valid".

This is an excerpt from Manning's book Deep Learning with Python. Login to get full access to this book.

If you want to get an output feature map with the same spatial dimensions as the input, you can use padding. Padding consists of adding an appropriate number of rows and columns on each side of the input feature map so as to make it possible to fit center convolution windows around every input tile. For a 3 × 3 window, you add one column on the right, one column on the left, one row at the top, and one row at the bottom. For a 5 × 5 window, you add two rows (see figure 5.6).

Figure 5.6. Padding a 5 × 5 input in order to be able to extract 25 3 × 3 patches

In Conv2D layers, padding is configurable via the padding argument, which takes two values: "valid", which means no padding (only valid window locations will be used); and "same", which means “pad in such a way as to have an output with the same width and height as the input.” The padding argument defaults to "valid".