liveBook · Manning

Chapter 13: Read-only data optimization

What you will learn in chapter 13 of Grokking GPU Programming

This introduction explains why repeated warp-wide reads can strain ordinary caches and how constant memory provides a dedicated cache for read-only data.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This introduction explains why repeated warp-wide reads can strain ordinary caches and how constant memory provides a dedicated cache for read-only data.

Key terms: warp, global memory, cache line, L1/L2 caches, constant memory, read-only, dedicated on-chip cache

13.1 One value, 32 reads

This section explains the cache costs of reading shared constants from global memory and distinguishes kernel arguments from explicitly stored constant data.

Key terms: warp, global memory, L1/L2 cache, DRAM, cache line, 32-byte sectors, per-pixel data, kernel argument, pointer, __constant__ array

13.2 How constant memory works

This section introduces constant memory and explains its size, host-write and thread-read roles, and performance under same-location versus different-location reads.

Key terms: constant memory, shared memory, host, serialization

13.3 Constant memory within the hierarchy

This section places constant memory in the GPU memory hierarchy and explains its cache, capacity, performance, and appropriate uses.

Key terms: constant memory, device DRAM, constant cache, streaming multiprocessors, __constant__, constant memory address space, working set, cache miss, L1, L2

13.4 The three steps for using constant memory

This section explains how to declare, populate, and read CUDA constant memory, including its distinction from const and the ordering of updates.

Key terms: __constant__, constant memory, constant cache, broadcast, const, global memory, cudaMemcpyToSymbol, symbol, host-to-device, default stream

13.5 The broadcast mechanism

This section explains constant-memory broadcast within a warp and contrasts it with opportunistic global-memory caching.

Key terms: warp, constant memory, broadcast, serialized, opportunistic broadcast, guaranteed broadcast, constant cache, read-only, cache-line fetch

13.6 How broadcast works, step by step

This section traces constant-memory broadcast within a warp and explains its behavior with inactive threads, cache misses, and multiple addresses.

Key terms: warp, constant cache, broadcast, transaction, active thread, miss, fetch, eviction, address, load instruction

13.7 The limitation of constant memory

This section explains why divergent constant-memory reads serialize, how cache locality affects their cost, and when global memory is preferable.

Key terms: constant cache, broadcast, unique addresses, serialized rounds, cache line, cache hit, scattered but clustered, global memory, scattered access, coalesce

13.8 Putting it all together: constant memory array scaling

This section applies the three-step constant-memory pattern to scaling an array and explains how to adapt the example for submission.

Key terms: constant memory, scaleFactor, cudaMemcpyToSymbol(), broadcast, warp, __constant__, kernel, main(), standard input, %g

13.9 Step 1: Declare in constant memory

This section explains how a file-scope declaration places scaleFactor in constant memory for cached, warp-wide access.

Key terms: file scope, __constant__, constant memory, warp, broadcast

13.10 Step 2: Copy data to constant memory from the host

This section explains why host-side copying uses a constant-memory symbol rather than a dynamically allocated device pointer.

Key terms: symbol name, scaleFactor, __constant__, statically allocated, cudaMalloc, cudaFree

13.11 Step 3: Use it in the kernel

This section traces an array-scaling kernel that broadcasts a constant scale factor and compares explicit constant memory with a scalar kernel argument.

Key terms: scaleFactor, constant memory, broadcast, global memory, idx, cudaMemcpyToSymbol, cudaMemcpy, constant variable, kernel argument, __constant__

13.12 Common mistakes

This section identifies three mistakes caused by treating constant memory like ordinary memory and states their consequences.

Key terms: constant memory, 64 KB, __constant__, cudaMemcpy, device pointer, stale values, const, global-memory speed

13.13 Mistake 1: Exceeding 64 KB

This section explains how to recognize and avoid exceeding the GPU’s constant-memory capacity.

Key terms: constant memory, __constant__, 64 KB, ptxas

13.14 Mistake 2: Using the wrong copy function

This section explains why copying to a constant-memory symbol requires cudaMemcpyToSymbol and how to update part of a constant array.

Key terms: cudaMemcpy, __constant__, cudaMemcpyToSymbol, invalid-argument error, offset, cudaMemcpyHostToDevice, cudaMemcpyDeviceToDevice

13.15 Mistake 3: Using const instead of __constant__

This section distinguishes CUDA constant memory from ordinary const declarations and shows how to correct the mistake.

Key terms: const, __constant__, constant memory, global memory, file scope, cudaMemcpyToSymbol(), constant region, broadcast, lookup table, cache line

13.16 Summary

Concepts explained in this chapter

← Chapter 12: Parallel matrix multiplication Chapter 14: Warps and occupancy →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.