A convolutional neural network (CNN) learns filters that recognize local patterns as they move across an input. For text, a filter slides across a sequence of words rather than across an image. This makes CNNs useful for finding word-order patterns that matter to tasks such as text classification and word tagging. section 7.1 section 7.2.1
Picture a stencil moving along a sentence. At each position, only the words under its opening contribute to a match score. A CNN replaces the stencil with numeric weights: training adjusts those weights toward patterns predictive of the target. Kernel width determines how many neighboring words the filter examines at once. section 7.2.1
A grammatical pattern detector
Suppose each word has channels indicating its part of speech. A width-three filter can score the pattern adverb → verb → noun. In a tagged quotation, a PyTorch `Conv1d` layer with hand-set weights gives “rightly timed pause” a score of 3. That small test makes the filter’s behavior inspectable before its weights are learned from data. section 7.2.6
Listing 7.8 Loading hardcoded weights into a Conv1d layer
>>> kernel = torch.tensor(kernel.values, dtype=torch.float32)
>>> kernel = kernel.unsqueeze(0) #1
>>> conv = torch.nn.Conv1d(in_channels=4,
... out_channels=1,
... kernel_size=3,
... bias=False)
>>> conv.load_state_dict({'weight': kernel})
>>> print(conv.weight)
tensor([[[1., 0., 0.],
[0., 0., 0.],
[0., 1., 0.],
[0., 0., 1.]]])
Inspect how the three-position, four-channel pattern is loaded as the weights of a PyTorch `Conv1d` layer. section 7.2.6
In a text classifier, token IDs become word embeddings, and embedding dimensions become convolution channels. Transpose the embedding output so `Conv1d` moves along the word sequence—not along the dimensions of each word vector. Several kernel widths can capture patterns of different lengths; activation, pooling and a linear layer then turn filter outputs into a prediction. section 7.4.2 section 7.5.1
Tensor shapes matter. With stride one and no padding, a length-K filter shortens a sequence of length N to N − K + 1. Pooling can reduce the remaining positions, while clipping and padding give examples a consistent input length. Clipping also sets a limit: patterns beyond the retained text cannot be detected. section 7.2.5 section 7.4.1 section 7.5.1
A practical exercise is to classify labeled disaster tweets with a PyTorch CNN. Check training and test accuracy separately: example training runs show test accuracy falling while training accuracy continues to rise. Tune kernel widths, dropout and other hyperparameters against held-out results rather than assuming that more training improves the classifier. section 7.5 section 7.5.1 section 7.7
