liveBook · Manning

Memory coalescing

As explained in GPU Programming with Triton

Memory coalescing is an access pattern in which neighboring parallel threads access adjacent memory locations so those accesses can be combined efficiently.

Memory coalescing combines the addresses requested by neighboring lanes in a warp into as few global-memory transactions as possible. Consecutive addresses use transactions efficiently; widely spaced addresses can fetch many bytes the lanes did not request. section 5.2.4

For a 32-lane warp loading FP32 values:

  • With consecutive addresses, the 128 requested bytes occupy four 32-byte sectors, so four transactions serve the warp. section 5.2.4
  • With a stride of 32 elements between lanes, the addresses fall in 32 separate sectors. The same 128 useful bytes require 32 transactions, using only 12.5% of the transferred bytes. section 5.2.4

A transpose store

In a naive matrix transpose, neighboring lanes can write to addresses separated by a row stride. For a large row-major output, each lane may touch a separate sector: 32 transactions for 128 useful bytes, rather than four for contiguous stores. Caching and architecture can affect traffic observed at DRAM, but the kernel still issues strided addresses. section 5.4.2

Figure 5.4 Memory transaction overhead. Four representative lanes are shown. For a complete FP32 warp, contiguous addresses occupy 4 adjacent 32-byte segments, while a sufficiently large stride can place the 32 requested values in 32 separate segments.
CH05 F04 Fartale

Compare how contiguous and widely spaced lane addresses occupy memory sectors. section 5.2.4

Correct indexing does not guarantee efficient data movement. Check the addresses requested together by a warp; in tiled transposes, reordering data on chip can help make global-memory accesses coalesced. section 5.10 section 5.4.2

How GPU Programming with Triton covers it

How deep it goes
Quantifies 32-lane FP32 accesses in 32-byte sectors, comparing contiguous and strided patterns.
Examples
Compares contiguous and stride-32 FP32 loads, then contrasts contiguous and strided transpose stores.
What you can do after
Evaluate warp access efficiency by comparing requested addresses with the memory sectors they occupy.
How it connects
Connects strides, warps, global memory, and matrix transpose to efficient access patterns.

Also explained in

Compare how 3 books explain memory coalescing →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.