Imagine many GPU threads scaling pixels by the same factor. Each thread needs the same value, so fetching it separately would repeat work. The constant cache lets threads share that read: when a warp requests one constant-memory address, the value can be broadcast to all participating threads. section 13.5 section 13.6
Constant memory is a 64 KB read-only address space in device DRAM. Its dedicated on-chip cache sits near each streaming multiprocessor (SM), the GPU’s processing unit. The 64 KB limit describes the address space, not the size of each SM’s cache. Device threads read constants; the host supplies their values. section 13.3 section 13.16
Figure 13.2 The GPU memory hierarchy as a classroom: each thread (student) has its own registers (sticky notes on the desk); threads in a block share that block’s shared memory (whiteboard); the per-SM L1 and constant caches and the GPU-wide L2 cache act as shelves; and device DRAM (the storage room) holds both global memory (the filing cabinet) and constant memory (the blackboard).

The classroom diagram places the constant cache near the processors and its backing constant memory in device DRAM. section 13.3
A warp is a group of 32 threads that issues a load together. If its active threads request the same address, one transaction serves them all; inactive threads do not participate. The cache can still miss, such as on a first read or when one constant evicts another. Dedicated broadcast means per-thread data cannot evict a constant from this cache, not that every read is a cache hit. section 13.6 section 13.5
Scaling pixels
Suppose every active thread multiplies its pixel by scaleFactor. Because they all read the same constant-memory address, the cache broadcasts one result across the warp. If the value were a three-float vector instead, each float would require a separate load and broadcast, though later loads could hit the same cache line. section 13.6
Different addresses change the picture. The cache serves one unique address per round, broadcasting each result to the threads that requested it. A warp requesting N distinct addresses needs N serialized rounds, up to 32. Addresses in the same cache line still require separate rounds, but after the line is fetched, later rounds can hit in the cache. section 13.7
A lookup-table trap
If one thread reads lookup[42], another reads lookup[7], and others choose their own indices, the reads may serialize even though the table is read-only. For a large read-only array or per-thread indexing, consider a const __restrict__ pointer instead of relying on constant-memory broadcast. section 13.7 section 13.16
To use constant memory in CUDA:
- Declare a device symbol with __constant__, then copy host values to it with cudaMemcpyToSymbol(). No cudaMalloc or device pointer is needed for the symbol. section 13.16 section 13.4
- Read the symbol in the kernel. Prefer a small array whose elements are read uniformly across a warp; for a single scalar, pass a kernel argument. section 13.16
- Do not confuse __constant__ with ordinary const: a const device pointer still refers to global memory and does not provide the dedicated constant cache or its broadcast. section 13.4
