liveBook · Manning

Column-major order

As explained in GPU Programming with Triton

Column-major order is a memory layout in which the elements of each column are stored contiguously before the next column.

Column-major order stores a matrix one column at a time. Elements in a column are adjacent in memory, so moving down a column advances by one position; moving across a row skips over the other rows. section 5.1

For a matrix with M rows and N columns:

  • The element at row i, column j has flat offset j × M + i. In row-major order, the corresponding offset is i × N + j. section 5.1
  • The logical matrix does not change between layouts; only the physical address of each element changes. Strides describe this mapping and can let a transposed view use the same storage. section 5.1 section 5.2.2
  • For GPU kernels, neighboring lanes requesting consecutive addresses can use memory transactions efficiently; strided accesses may transfer more unused data. section 5.10

A 3 × 4 matrix

Column-major storage lists [a00, a10, a20, a01, a11, a21, a02, a12, a22, a03, a13, a23]. Thus a21, at row 2 and column 1, has offset 1 × 3 + 2 = 5. section 5.1

Figure 5.1 Row-major versus column-major memory layout. In row-major order (top), elements in the same row are adjacent in memory. In column-major order (bottom), elements in the same column are adjacent.
CH05 F01 Fartale

Compare which matrix direction is contiguous in each layout. section 5.1

How GPU Programming with Triton covers it

How deep it goes
Defines the layout, gives its flat-index formula, and connects contiguous access to memory behavior.
Examples
Uses a 3 × 4 matrix, its flat storage sequence, and language conventions to illustrate the layout.
What you can do after
Understand how to map matrix coordinates to flat offsets and assess which direction accesses contiguous memory.
How it connects
Connects column-major order to strides, transposed views, and memory coalescing.

Also explained in

Compare how 3 books explain column-major order →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.