Padding adds rows and columns around an input feature map, or special tokens to a sequence, so fixed-size computations can include boundaries or run efficiently in batches. section 8.1.1 section 14.5
In convolutional image models, padding addresses border effects:
- A 3 × 3 window applied without padding to a 5 × 5 feature map has only 9 centered locations, producing a 3 × 3 output. section 8.1.1
- Adding one row and column on every side lets a 3 × 3 window cover every original tile and preserve spatial dimensions. section 8.1.1
- Conv2D uses padding = "valid" for no padding and padding = "same" for an output with the input's width and height; "valid" is the default. section 8.1.1
Figure 8.6 Padding a 5 × 5 input in order to be able to extract 25 3 × 3 patches
Shows how padding a 5 × 5 input enables extraction of 25 centered 3 × 3 patches. section 8.1.1
In text classification, padding makes variable-length inputs rectangular:
- Tokenized sentences receive a special "[PAD]" token, usually at the end, until they reach a chosen sequence length. section 14.5
- Padding allows batch calculations to be parallelized efficiently on a GPU, but padding tokens must not degrade model predictions. section 14.5
- IMDb reviews can be truncated or padded to 600 words; the resulting batches have shape (batch_size, sequence_length), with many trailing zeros. section 14.5
Padding IMDb reviews in R/Keras
max_length <- 600 text_vectorization <- layer_text_vectorization( output_mode = "int", output_sequence_length = max_length ) text_vectorization |> adapt(train_ds_no_labels) sequence_train_ds <- train_ds |> dataset_map(\(x, y) tuple(text_vectorization(x), y))
