Memory alignment places data at addresses that meet a type or hardware boundary. In CUDA, a coalesced warp access can still need an extra transaction if its starting address is not aligned to a 128-byte boundary. section 10.13
Check alignment when a kernel’s memory traffic falls short of expectations:
- cudaMalloc returns pointers aligned to at least 256 bytes, but pointer arithmetic or a sub-array offset can misalign the address passed to a kernel. section 10.13
- A float3 occupies 12 bytes without padding, so its elements do not stay on 16-byte boundaries. Pad to float4 when reading whole elements, or use separate x, y, and z arrays (SoA) when reading individual fields. section 10.13
- When carving multiple arrays from dynamic shared memory, align each region’s offset to a multiple of its type’s alignof(T). section 11.2
A one-float offset
Passing data + 1 starts access one float into the allocation and can misalign a warp’s access. Passing the aligned base avoids that offset; if the algorithm requires a sub-array, pad the allocation so the sub-array begins on a 128-byte boundary. section 10.13
Table 10.4 The chapter’s pitfalls at a glance, symptom first: start from what you measure, and it points at the mistake to suspect.
|
What you
see
|
Likely
cause
|
First check
|
|
Math-heavy
kernel runs no faster than a trivial one
|
Below the
effective ridge point (Mistake 1)
|
Compute
ops/byte; compare against the ridge point for your actual access pattern
|
|
Coalesced
kernel still short of expected bandwidth
|
Warp start
address misaligned (Mistake 2)
|
Check
pointer arithmetic and sub-array offsets against the 128-byte boundary
|
|
float3
kernel far slower than its float4 or SoA twin
|
Unpadded
12-byte elements (Mistake 2’s float3 case)
|
Pad to
float4 or convert to SoA
|
Use the symptom-first table to check whether low bandwidth points to a misaligned start or an unpadded float3 layout. section 10.13




