Memory coalescing
Memory coalescing is a GPU access pattern in which nearby threads access adjacent memory locations so their requests can be combined into fewer memory transactions.
CUDA for LLMs
Memory coalescing is a GPU memory-access technique in which neighboring threads access neighboring memory locations so requests can be combined.
- How it is used
- Uses coalescing to fix bandwidth-wasting access patterns in transformer kernels before further optimization.
- How deep it goes
- Traces warp addresses and thread mappings through cache-line behavior, kernel code, and measured performance, then extends the approach to tiled loads.
- Examples
- Works through row-wise GEMV, thread-remapped GEMM, and fused softmax.
- What you can do after
- Apply it via CUDA thread remapping and evaluate the result with realistic-size profiling.
GPU Programming with Triton
Memory coalescing is an access pattern in which neighboring parallel threads access adjacent memory locations so those accesses can be combined efficiently.
- How it is used
- Analyzes how warp-level global-memory access patterns affect transaction counts and GPU performance.
- How deep it goes
- Quantifies 32-lane FP32 accesses in 32-byte sectors, comparing contiguous and strided patterns.
- Examples
- Compares contiguous and stride-32 FP32 loads, then contrasts contiguous and strided transpose stores.
- What you can do after
- Evaluate warp access efficiency by comparing requested addresses with the memory sectors they occupy.
Read how GPU Programming with Triton explains memory coalescing →
Grokking GPU Programming
Memory coalescing combines memory requests from GPU threads into fewer transactions when their accesses are suitably grouped.
- How it is used
- Reduces global-memory bottlenecks by grouping simultaneous GPU thread requests into fewer transactions.
- How deep it goes
- Explains warp-level access snapshots and transaction counting, including the distinction between cache-line and sector models.
- Examples
- Uses an array copy, row- and column-wise matrix access, and a tiled matrix transpose.
- What you can do after
- Apply warp-level address snapshots to identify uncoalesced accesses and choose thread assignments or shared-memory tiling.
Read how Grokking GPU Programming explains memory coalescing →
Also covered in
What to know first
- GPU memoryGPU memory is memory available to a graphics processing unit for storing data and state used in graphics or computation.
- memory transactionA memory transaction is a hardware operation that transfers a unit or segment of data between memory and a processor, potentially combining requests from multiple lanes.
- shared memoryShared memory is a memory region accessible to multiple processes, threads, or other concurrent execution contexts for sharing data.
- warpA warp is a group of GPU threads scheduled and issued together for execution, typically processing the same instruction on different data.
Related concepts
- memory transactionuses · A memory transaction is a hardware operation that transfers a unit or segment of data between memory and a processor, potentially combining requests from multiple lanes.
- multi-kernel approachused by · A multi-kernel approach divides a computation into multiple separately launched GPU kernels, typically assigning separate stages to separate launches.
- scattered memory access patterncontrasts with · A scattered memory access pattern occurs when threads access memory locations that are not close together, often requiring separate memory transactions.
Questions about memory coalescing
What is memory coalescing?
Memory coalescing is a GPU access pattern in which nearby threads access adjacent memory locations so their requests can be combined into fewer memory transactions.
Which Manning books explain memory coalescing?
CUDA for LLMs, GPU Programming with Triton and Grokking GPU Programming.
What should I know before learning memory coalescing?
Manning books that teach memory coalescing usually introduce GPU memory, memory transaction, shared memory and warp first.
What is the difference between memory coalescing and scattered memory access pattern?
Memory coalescing is a GPU access pattern in which nearby threads access adjacent memory locations so their requests can be combined into fewer memory transactions. A scattered memory access pattern occurs when threads access memory locations that are not close together, often requiring separate memory transactions.
What is memory coalescing also called?
Memory coalescing is also called coalesced access or coalesced memory access.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


