A convolution kernel, or filter, is a small array of weights that slides across numerical input. At each position, its weights combine with the values under them to produce one output. The resulting sequence shows where a local pattern appears. Think of a stencil moved across a line of text: it reveals a match wherever its openings line up. section 7.2.1 section 7.2.5
In natural language processing, a kernel can look for an ordered pattern rather than merely the presence of words. For example, a two-position pattern can mark an adjective followed by a noun: “right ones” and “right order” match, while “ones in” does not. This simple part-of-speech stencil returns matches; learned kernels can instead use numerical word representations to detect patterns useful for a prediction task. section 7.2.2 section 7.4.3
Build a two-value kernel
A kernel of [0.5, 0.5] takes the average of each adjacent pair. This pure Python version makes the sliding calculation visible: kernel = [0.5, 0.5] inpt = [0, 1, 1, 0] output = [sum(inpt[i+k] * kernel[k] for k in range(len(kernel))) for i in range(len(inpt) - len(kernel) + 1)] print(output) # [0.5, 1.0, 0.5] With no padding and a one-position stride, an input of length N and kernel of length K produces N − K + 1 values. The middle result is highest because both values under the kernel are 1. section 7.2.5 section 7.4.1
For text represented by several values per word, a kernel spans both the word positions and the representation’s channels. A three-word pattern over four part-of-speech channels uses a 4 × 3 weight matrix. A hand-set kernel for adverb–verb–noun produces a strong response over “rightly timed pause.” Such a response is a weighted score, not necessarily a yes-or-no answer. section 7.2.6
Listing 7.8 Loading hardcoded weights into a Conv1d layer
>>> kernel = torch.tensor(kernel.values, dtype=torch.float32)
>>> kernel = kernel.unsqueeze(0) #1
>>> conv = torch.nn.Conv1d(in_channels=4,
... out_channels=1,
... kernel_size=3,
... bias=False)
>>> conv.load_state_dict({'weight': kernel})
>>> print(conv.weight)
tensor([[[1., 0., 0.],
[0., 0., 0.],
[0., 1., 0.],
[0., 0., 1.]]])
Inspect how the 4 × 3 hand-set pattern becomes the weight tensor of a PyTorch Conv1d layer. section 7.2.6
Known patterns can be designed directly. A Morse-code dot detector, for instance, uses negative weights for silence around positive weights for a short tone. For less predictable language patterns, training adjusts filter weights through backpropagation instead. Word embeddings supply dense input channels, but their dimensions must be transposed to align with Conv1d channels so the kernel moves along the word sequence. section 7.3.1 section 7.4.2 section 7.4.3
Kernel size and stride affect output length, so check tensor shapes before connecting convolution to pooling and classification layers. Padding and clipping can give texts a consistent input length; global max pooling can reduce a convolution’s output to one value per channel. A high training score alone is not proof of a dependable detector: test performance can fall as training continues. section 7.5.1 section 7.4.1
