Imagine an image filter where every GPU thread needs the same small set of coefficients. Constant memory gives those read-only values a dedicated cache, so a group of threads can receive the same value together instead of competing with per-thread data for ordinary cache space. section 13.0 section 13.3
Constant memory is a 64 KB device-wide address space backed by device DRAM, with a dedicated cache near each streaming multiprocessor (SM). GPU threads can read it but cannot write it; the host supplies its values. The 64 KB address space is not the size of each SM’s cache. section 13.3 section 13.2
Threads execute in groups of 32 called warps. When the active threads in a warp request the same constant-memory address, one fetch broadcasts the value to them. Global-memory reads of the same address can also be combined, but their cached data competes with other data and can be evicted. section 13.5 section 13.11
Use constant memory in three steps:
- Declare a variable or array at file scope with __constant__. This reserves a named slot; it does not yet supply the host value. section 13.4
- Before launching the kernel, copy the host data with cudaMemcpyToSymbol(), using the symbol name rather than a cudaMalloc device pointer. section 13.10 section 13.4
- Read the symbol directly in the kernel. Its value persists across launches until the host overwrites it; there is no constant-memory allocation to free. section 13.11 section 13.16
__constant__ float SCALE; // In GPU constant memory (64 KB region)
// In host code: copy to constant memory before launching kernel
float h_scale = 2.5f;
cudaMemcpyToSymbol(SCALE, &h_scale, sizeof(float)); // not cudaMemcpy
// In kernel: read SCALE directly - hardware broadcasts to all 32 threads
__global__ void kernel(float* data, int n) {
int idx = blockIdx.x * blockDim.x + threadIdx.x;
if (idx < n) data[idx] *= SCALE; // One broadcast fetch serves all 32 threads
}This example declares a constant symbol, copies a host value into it, and has each active thread multiply its own array element by that shared value. section 13.15
Scaling four values
With a scale factor of 2.5 and input [1.0, 2.0, 3.0, 4.0], four active threads read the same constant and produce [2.5, 5.0, 7.5, 10.0]. Each thread accesses a different input element, but the factor can be broadcast to all four. section 13.11
Different addresses change the calculation. If a warp requests N distinct constant-memory addresses, the requests take N serialized rounds, up to 32. Addresses in the same cache line can make later rounds cache hits, but they do not remove the serialization. For a large read-only array or per-thread indexing, consider a const __restrict__ pointer instead. section 13.7 section 13.16
Watch for these common mistakes:
- Exceeding the 64 KB region can make the build fail. Add up the sizes of constant declarations and leave room for other constants. section 13.13
- Using cudaMemcpy as if the symbol were a device pointer can leave stale values. Use cudaMemcpyToSymbol with the symbol name. section 13.12 section 13.10
- A plain const or const pointer does not place data in the constant-memory region. Use __constant__ when that region and its broadcast are needed. section 13.15
For a single scalar, a kernel argument is often simpler: CUDA stores kernel arguments in constant memory automatically. An explicit __constant__ array is useful for small, repeatedly reused values that threads usually read at the same index. section 13.1 section 13.16
