A stride is the number of input positions a convolution kernel skips between successive applications. With stride 1, a two-token kernel checks every adjacent pair; with stride 2, it checks alternating positions. section 7.2.2
Stride changes both coverage and output length:
- Stride 1 preserves the most opportunities to detect a pattern and is the common choice for kernels of size two or more. section 7.2.2
- Stride 2 reduces the number of examined spans, so some matches can be skipped; in the paired-word example, it detected both matches only by chance. section 7.2.2
- Larger strides shorten the convolution output sequence; the amount of shortening depends on stride, kernel size, padding, and dilation. section 7.5.1
Checking part-of-speech pairs
A two-token stencil over “right ones in the right order” finds matches at spans 0–1 and 4–5 with stride 1. Moving by stride 2 inspects spans 0–1, 2–3, and 4–5, omitting the intervening pairs. section 7.2.2
def calc_conv_out_seq_len(seq_len, kernel_len,
stride=1, dilation=1, padding=0):
"""
L_out = (L_in + 2 * padding - dilation * (kernel_size - 1) - 1)
1 + _______________________________________________________
stride
"""
return (
1 + (seq_len +
2 * padding - dilation * (kernel_len - 1) - 1
) //
stride
)
Use the convolution output-length calculation to inspect how stride changes the sequence shape. section 7.5.1
