liveBook · Manning

Alignment

As explained in Grokking GPU Programming

Memory alignment is the placement of data at an address satisfying the boundary requirements of its type or hardware access.

Memory alignment places data at addresses that meet a type or hardware boundary. In CUDA, a coalesced warp access can still need an extra transaction if its starting address is not aligned to a 128-byte boundary. section 10.13

Check alignment when a kernel’s memory traffic falls short of expectations:

  • cudaMalloc returns pointers aligned to at least 256 bytes, but pointer arithmetic or a sub-array offset can misalign the address passed to a kernel. section 10.13
  • A float3 occupies 12 bytes without padding, so its elements do not stay on 16-byte boundaries. Pad to float4 when reading whole elements, or use separate x, y, and z arrays (SoA) when reading individual fields. section 10.13
  • When carving multiple arrays from dynamic shared memory, align each region’s offset to a multiple of its type’s alignof(T). section 11.2

A one-float offset

Passing data + 1 starts access one float into the allocation and can misalign a warp’s access. Passing the aligned base avoids that offset; if the algorithm requires a sub-array, pad the allocation so the sub-array begins on a 128-byte boundary. section 10.13

Table 10.4 The chapter’s pitfalls at a glance, symptom first: start from what you measure, and it points at the mistake to suspect.
What you see
Likely cause
First check
Math-heavy kernel runs no faster than a trivial one
Below the effective ridge point (Mistake 1)
Compute ops/byte; compare against the ridge point for your actual access pattern
Coalesced kernel still short of expected bandwidth
Warp start address misaligned (Mistake 2)
Check pointer arithmetic and sub-array offsets against the 128-byte boundary
float3 kernel far slower than its float4 or SoA twin
Unpadded 12-byte elements (Mistake 2’s float3 case)
Pad to float4 or convert to SoA

Use the symptom-first table to check whether low bandwidth points to a misaligned start or an unpadded float3 layout. section 10.13

How Grokking GPU Programming covers it

How deep it goes
Explains 128-byte warp starts, pointer offsets, float3 layout, and shared-memory offsets without developing alignment beyond these cases.
Examples
Uses a one-float pointer offset, float3 arrays, and mixed-type shared-memory regions.
What you can do after
Diagnose alignment-related bandwidth losses and choose pointer, padding, or layout fixes.
How it connects
Connects alignment with memory coalescing, cache lines, vector loads, AoS, SoA, and shared memory.

Also explained in

Compare how 5 books explain alignment →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.