liveBook · Manning

Row-major order

Also called row-major layout

Row-major order is a multidimensional array layout in which the rightmost index varies fastest, so the elements of each row are contiguous in memory.

How 2 Manning books explain it

GPU Programming with Triton

Row-major order is a memory layout in which the elements of each row are stored contiguously before the next row.

How it is used
Uses row-major layout to calculate matrix addresses and reason about GPU memory access.
How deep it goes
Defines contiguous row storage and works through flat-offset and stride calculations, including noncontiguous tensor views.
Examples
Works through a 3 × 4 matrix and calculates the flat offset and strides for element (2, 1).
What you can do after
Apply the flat-index formula to locate matrix elements in contiguous row-major storage.

Read how GPU Programming with Triton explains row-major order →

Grokking GPU Programming

Row-major order is a multidimensional array layout in which elements of each row are contiguous in memory.

How it is used
Uses row-major layout to reason about matrix indexing, GPU memory access, and library interoperability.
How deep it goes
Works through flattening, indexing errors, warp access patterns, shared-memory padding, and the cuBLAS transpose trick.
Examples
Uses a 2×3 matrix, row and column sums, padded shared-memory tiles, and a cuBLAS matrix multiplication call.
What you can do after
Diagnose/debug indexing and access-pattern problems using row-major offsets and small hand-checked matrices.

Read how Grokking GPU Programming explains row-major order →

Also covered in

What to know first

Related concepts

Questions about row-major order

What is row-major order?

Row-major order is a multidimensional array layout in which the rightmost index varies fastest, so the elements of each row are contiguous in memory.

Which Manning books explain row-major order?

GPU Programming with Triton and Grokking GPU Programming.

What should I know before learning row-major order?

Manning books that teach row-major order usually introduce array, C++, Fortran and shared memory first.

What is the difference between row-major order and column-major order?

Row-major order is a multidimensional array layout in which the rightmost index varies fastest, so the elements of each row are contiguous in memory. Column-major order is a multidimensional array layout in which the leftmost dimension varies fastest, making each matrix column contiguous in memory.

What is row-major order a kind of?

Row-major order is a kind of memory layout. Memory layout is the arrangement and mapping of data elements or regions to locations in memory.

What is row-major order also called?

Row-major order is also called row-major layout.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.