Row-major order stores a matrix one row at a time, making neighboring elements within each row adjacent in memory. For a contiguous matrix with N columns, element (i, j) is at flat offset i × N + j. section 5.1
A 3 × 4 matrix
Rows occupy offsets 0–3, 4–7, and 8–11. The element at (2, 1) is at offset 2 × 4 + 1 = 9; the corresponding strides are (4, 1). section 5.2.1
In a GPU kernel, row-major layout makes row starts easy to calculate: row i begins at i × N. But shape alone may not describe a tensor’s physical layout; transposes and slices can change its strides, so kernels may need runtime stride values. section 3.5.3 section 5.2.1
Figure 5.1 Row-major versus column-major memory layout. In row-major order (top), elements in the same row are adjacent in memory. In column-major order (bottom), elements in the same column are adjacent.
Compare which matrix direction is contiguous in row-major and column-major layouts. section 5.1

