liveBook · Manning

Scattered memory access pattern

As explained in Grokking GPU Programming

A scattered memory access pattern occurs when threads access memory locations that are not close together, often requiring separate memory transactions.

A warp has scattered memory access when its threads request unrelated locations. Unlike a regular stride, these addresses may not share cache lines, so the hardware cannot combine their requests. section 10.7

In the 128-byte cache-line teaching model, 32 scattered float loads can require 32 transactions: 4,096 bytes fetched for 128 useful bytes, about 3% utilization. Modern GPUs count 32-byte sectors, making the corresponding figure about 12.5%; these figures compare access patterns, not runtime. section 10.7 section 10.4

__global__ void copyScattered(float* input, float* output,
                              int* indices, int n) {
    int idx = blockIdx.x * blockDim.x + threadIdx.x;
    if (idx < n) {
        output[idx] = input[indices[idx]];    #A
    }
}

This indexed copy can produce scattered reads when neighboring threads’ indices refer to unrelated memory locations. section 10.7

Reverse order is not necessarily scattered

With consecutive idx values, input[n - 1 - idx] still accesses consecutive addresses, just in reverse. Reversing direction alone does not make accesses scattered. section 10.7

How Grokking GPU Programming covers it

How deep it goes
Quantifies transaction waste with a 128-byte cache-line model and notes the differing 32-byte-sector accounting.
Examples
Uses an indexed copy and reverse indexing to distinguish unrelated addresses from consecutive addresses in reverse order.
What you can do after
Evaluate a warp’s requests to recognize scattered accesses and estimate their transaction waste.
How it connects
Contrasts scattered access with coalesced and strided access, and relates transaction waste to memory bandwidth.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.