liveBook · Manning

Stride

Stride is the step or spacing between successive positions in an array access or successive placements of an operation over input data.

How 11 Manning books explain it

Grokking GPU Programming

Memory stride is the distance between memory locations accessed by successive elements of an access pattern.

How it is used
Uses stride to describe both neighboring-thread memory accesses and partner distances in parallel reduction and scan.
How deep it goes
Works through stride sequences, access efficiency, and shared-memory bank mapping, including alignment and element-size qualifications.
Examples
Traces an eight-element reduction and scan, stride-2 warp reads, and particle field layouts.
What you can do after
Diagnose memory access spacing and evaluate layout changes while checking scan boundaries and synchronization.

Read how Grokking GPU Programming explains stride →

GPU Programming with Triton

The number of storage elements between successive logical elements along an array or tensor dimension.

How it is used
Uses strides to map logical tensor indices to physical offsets in GPU kernels.
How deep it goes
Derives the multidimensional offset formula and explains views, unit and zero strides, runtime indexing, and coalescing.
Examples
Works through matrix offsets, a shared-storage transpose, and distinct input and angle row strides in a fused RoPE kernel.
What you can do after
Apply it via Triton by passing runtime strides, calculating offsets, and checking bounds and write coverage.

Read how GPU Programming with Triton explains stride →

Math and Architectures of Deep Learning

The step size with which a convolutional kernel moves across the input, affecting the output size in convolution and transposed convolution.

How it is used
Controls convolution and pooling stepping to determine which local positions produce output values and how feature maps are downsampled.
How deep it goes
Derives 1D output size from kernel bounds, then extends stepping to multidimensional and transposed convolutions.
Examples
Uses length-7 1D averaging windows, 3D video-volume stepping, PyTorch Conv1d calls, and 2 × 2 stride-2 pooling.
What you can do after
Apply it via PyTorch convolution or pooling parameters and calculate resulting convolution or transposed-convolution output sizes.

Read how Math and Architectures of Deep Learning explains stride →

Learn Generative AI with PyTorch

The step-size parameter that controls how far a convolutional operation moves and, in transposed convolution, how much upsampling occurs.

How it is used
Controls convolutional scanning and spatial downsampling in image-processing networks.
How deep it goes
Explains movement, output-size effects, padding interaction, numerical calculations, and VAE tensor-shape changes.
Examples
Uses a 3 × 3 image, fixed 2 × 2 weights, stride 2, padding 1, and a stride-based VAE encoder.
What you can do after
Apply stride via PyTorch Conv2d arguments and predict its effects on feature-map dimensions.

Read how Learn Generative AI with PyTorch explains stride →

Fast Python: High performance techniques for large datasets

How it is used
Explains how NumPy locates elements in views and supports memory-efficient array transformations.
How deep it goes
Defines byte steps per dimension and works through transposes, reversals, and slices; defers allocation details.
Examples
Uses reshaped and transposed integer arrays, sampled slices, and image flips, rotations, and crops.
What you can do after
Evaluate whether common NumPy transformations produce views and interpret their shape and stride metadata.

Read how Fast Python: High performance techniques for large datasets explains stride →

Deep Learning with PyTorch, Second Edition

Tensor metadata describing its shape and how elements are laid out and indexed in storage, including step size between positions.

How it is used
Explains how PyTorch tensors index storage and why views, slicing, transposition, and reshaping can avoid data copies.
How deep it goes
Derives the indexing formula and follows stride through views, transposes, reshaping, and contiguity, stopping short of deeper memory-system details.
Examples
Uses point tensors, indexed subtensors, transposes, contiguous copies, daily bike sequences, and size-offset-stride exercises.
What you can do after
Diagnose tensor layouts by predicting stride, storage offset, shared storage, and contiguity after indexing or reshaping.

Read how Deep Learning with PyTorch, Second Edition explains stride →

Grokking AI Algorithms, Second Edition

A stride is the spatial step size by which a convolutional filter moves across an input.

How it is used
Shrinks convolutional feature maps by increasing the distance between successive filter positions.
How deep it goes
Explains filter movement, padding, and output-size calculation through an 8×8-to-4×4 example and implementation.
Examples
Uses a 4×4 filter with stride 2 to reduce eight 8×8 feature maps to 4×4 maps.
What you can do after
Build it from scratch by configuring stride and calculating output dimensions in a downsampling convolution.

Read how Grokking AI Algorithms, Second Edition explains stride →

CUDA for LLMs

A memory access stride is the address distance between successive accesses in an access pattern.

How it is used
Explains how thread access spacing affects memory coalescing and how tensor layouts map logical coordinates to memory.
How deep it goes
Traces address spacing to cache-line transactions and bandwidth loss, while deferring the coalescing fix.
Examples
Uses GEMV thread addresses with N = 8192 and a 128 × 64 row-major layout with strides (64, 1).
What you can do after
Diagnose uncoalesced access by tracking thread addresses and calculating the stride between consecutive accesses.

Read how CUDA for LLMs explains stride →

Build a Large Language Model (From Scratch)

The step size by which the sliding window advances between sampled training examples.

How it is used
Controls how training contexts are sampled from tokenized text for GPT-style input-target pairs.
How deep it goes
Defines stride and implements it in the dataset loop, including overlap behavior and context-size interactions.
Examples
Demonstrates stride values of 1, 2, and 4 with context sizes of 2, 4, and 8.
What you can do after
Apply it via GPTDatasetV1 and DataLoader to control overlap among sampled training contexts.

Read how Build a Large Language Model (From Scratch) explains stride →

Natural Language Processing in Action, Second Edition

A stride is the number of input positions by which a convolution kernel moves between successive applications.

How it is used
Uses stride to control how convolution kernels scan token sequences and produce pattern-location maps.
How deep it goes
Explains stride through adjacent-token experiments and output-length formulas, but stops short of broader optimization guidance.
Examples
Demonstrates stride with part-of-speech bigrams and PyTorch Conv1d configurations using stride 1.
What you can do after
Apply stride by selecting a scan step and calculating the resulting convolution sequence length.

Read how Natural Language Processing in Action, Second Edition explains stride →

Python How-To

How it is used
Uses stride to select spaced elements from a sequence and to replace selected positions in a mutable list.
How deep it goes
Explains stride in slice notation, positive and negative directions, and list replacement at strided positions.
Examples
Uses list slices to select every third element, reverse a list, and replace every second item.
What you can do after
Apply Python slice notation to select spaced elements or replace selected positions in a list.

Read how Python How-To explains stride →

Also covered in

What to know first

Related concepts

Questions about stride

What is stride?

Stride is the step or spacing between successive positions in an array access or successive placements of an operation over input data.

Which Manning books explain stride?

Grokking GPU Programming, GPU Programming with Triton, Math and Architectures of Deep Learning, Learn Generative AI with PyTorch, Fast Python: High performance techniques for large datasets, Deep Learning with PyTorch, Second Edition, Grokking AI Algorithms, Second Edition, CUDA for LLMs and 3 more.

What should I know before learning stride?

Manning books that teach stride usually introduce array, tensor, linear layer and convolutional layer first.

What is the difference between stride and tensor shape?

Stride is the step or spacing between successive positions in an array access or successive placements of an operation over input data. A tensor shape is an ordered sequence of dimension sizes specifying the extent of a tensor along each axis.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.