liveBook · Manning

Constant memory

As explained in Grokking GPU Programming

Constant memory is a read-only CUDA memory space backed by device DRAM, with a limited address space and a dedicated cache near the streaming multiprocessors.

Imagine an image filter where every GPU thread needs the same small set of coefficients. Constant memory gives those read-only values a dedicated cache, so a group of threads can receive the same value together instead of competing with per-thread data for ordinary cache space. section 13.0 section 13.3

Constant memory is a 64 KB device-wide address space backed by device DRAM, with a dedicated cache near each streaming multiprocessor (SM). GPU threads can read it but cannot write it; the host supplies its values. The 64 KB address space is not the size of each SM’s cache. section 13.3 section 13.2

Threads execute in groups of 32 called warps. When the active threads in a warp request the same constant-memory address, one fetch broadcasts the value to them. Global-memory reads of the same address can also be combined, but their cached data competes with other data and can be evicted. section 13.5 section 13.11

Use constant memory in three steps:

  • Declare a variable or array at file scope with __constant__. This reserves a named slot; it does not yet supply the host value. section 13.4
  • Before launching the kernel, copy the host data with cudaMemcpyToSymbol(), using the symbol name rather than a cudaMalloc device pointer. section 13.10 section 13.4
  • Read the symbol directly in the kernel. Its value persists across launches until the host overwrites it; there is no constant-memory allocation to free. section 13.11 section 13.16
__constant__ float SCALE;               // In GPU constant memory (64 KB region)
 
// In host code: copy to constant memory before launching kernel
float h_scale = 2.5f;
cudaMemcpyToSymbol(SCALE, &h_scale, sizeof(float)); // not cudaMemcpy
 
// In kernel: read SCALE directly - hardware broadcasts to all 32 threads
__global__ void kernel(float* data, int n) {
    int idx = blockIdx.x * blockDim.x + threadIdx.x;
    if (idx < n) data[idx] *= SCALE;  // One broadcast fetch serves all 32 threads
}

This example declares a constant symbol, copies a host value into it, and has each active thread multiply its own array element by that shared value. section 13.15

Scaling four values

With a scale factor of 2.5 and input [1.0, 2.0, 3.0, 4.0], four active threads read the same constant and produce [2.5, 5.0, 7.5, 10.0]. Each thread accesses a different input element, but the factor can be broadcast to all four. section 13.11

Different addresses change the calculation. If a warp requests N distinct constant-memory addresses, the requests take N serialized rounds, up to 32. Addresses in the same cache line can make later rounds cache hits, but they do not remove the serialization. For a large read-only array or per-thread indexing, consider a const __restrict__ pointer instead. section 13.7 section 13.16

Watch for these common mistakes:

  • Exceeding the 64 KB region can make the build fail. Add up the sizes of constant declarations and leave room for other constants. section 13.13
  • Using cudaMemcpy as if the symbol were a device pointer can leave stale values. Use cudaMemcpyToSymbol with the symbol name. section 13.12 section 13.10
  • A plain const or const pointer does not place data in the constant-memory region. Use __constant__ when that region and its broadcast are needed. section 13.15

For a single scalar, a kernel argument is often simpler: CUDA stores kernel arguments in constant memory automatically. An explicit __constant__ array is useful for small, repeatedly reused values that threads usually read at the same index. section 13.1 section 13.16

How Grokking GPU Programming covers it

How deep it goes
Explains placement, caching, broadcast and serialized access, then walks through declaration, host copying, kernel reads, and common mistakes.
Examples
Works through four-element scaling and Gaussian filter coefficients, and contrasts uniform reads with indexed lookup-table accesses.
What you can do after
Apply it via CUDA using __constant__ and cudaMemcpyToSymbol, while choosing access patterns that favor broadcast.
How it connects
Connects constant memory with global memory, shared memory, caches, warps, and kernel arguments.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.