Stride is the number of token positions a sliding window advances when creating successive training examples. Smaller strides create overlapping contexts; setting stride equal to the context size prevents overlap. section 2.6
Worked sampling example
With max_length=4 and stride=1, the first input contains four token IDs and the next input begins one position later. With stride=4, successive four-token inputs do not overlap, allowing the dataset to use the text without skipping words.
