A matrix may be two-dimensional logically, but its data occupies a linear sequence of memory locations. Memory layout maps each logical index to a physical location. That mapping determines both which values a kernel reads and how efficiently it can move them. section 5.1 section 5.10
For a tensor with indices (i0, …, in−1) and element strides (s0, …, sn−1), the offset from the data buffer is i0×s0 + … + in−1×sn−1. Each stride is the number of elements advanced when its index increases by one. Adding the offset to the base pointer identifies the requested element. section 5.2
Two common contiguous matrix layouts differ in which direction is adjacent in memory:
- Row-major order stores each row consecutively. For a matrix with N columns, element (i, j) has offset i×N + j. section 5.1
- Column-major order stores each column consecutively, making elements in the same column adjacent. section 5.1
Figure 5.1 Row-major versus column-major memory layout. In row-major order (top), elements in the same row are adjacent in memory. In column-major order (bottom), elements in the same column are adjacent.
Compare how the same logical matrix maps to consecutive locations in row-major and column-major order. section 5.1
Shape alone does not determine the mapping. Views, slices, transposes, and broadcasting can change how indices address existing storage without copying it. Passing strides to a kernel lets its address calculations follow the actual layout rather than assume a fixed order. section 5.10 section 5.5.2
Rotation with split feature pairs
For a row of width D, a split layout pairs feature i with feature i+D/2. With shared rotation frequencies, the cosine and sine tables have row width D/2. A kernel must use the appropriate row width for each input: an offset error can load the wrong features or rotation values. section 3.5.2
Correct addresses are necessary but not sufficient for speed. On a GPU, memory coalescing depends on addresses requested together by neighboring lanes in a warp. Consecutive addresses use fewer global-memory transactions than strided accesses, which transfer more unused data. A tiled transpose can stage reordering on chip to improve access patterns; boundary masks protect loads and stores at incomplete tiles. section 5.10
Matrix multiplication access
In a naive matrix multiplication kernel with row-major inputs, a strip from a row of A advances through adjacent elements, while a strip down a column of B advances by its row stride. Each output program also loads its inputs independently. Tiling allows loaded values to contribute to multiple outputs, improving reuse. section 5.5.2 section 5.10
The level of layout control depends on the programming tool. Triton lets the programmer choose tile shapes and grid division while the compiler makes other decisions; Gluon exposes more direct control over tile arrangement and scheduling, at a portability cost across GPU generations. section 1.2.2




