liveBook · Manning

Chapter 11: Shared memory

What you will learn in chapter 11 of Grokking GPU Programming

This introduction explains why shared memory can speed up GPU computation and identifies the common collaboration pattern used in earlier kernels.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This introduction explains why shared memory can speed up GPU computation and identifies the common collaboration pattern used in earlier kernels.

Key terms: shared memory, global memory, reduction, prefix scans, histogram

11.1 When your kernel is slow

This section explains the GPU memory hierarchy and why reusing data in shared memory can reduce latency, along with its scope and resource limits.

Key terms: registers, shared memory, L1 cache, L2 cache, global memory, DRAM, latency, L2 hit, block, SRAM

11.2 Introducing shared memory

Introduces shared memory’s block-local uses, lifetime, declaration methods, and safe allocation of multiple arrays.

Key terms: shared memory, SM, block, threads, data reuse, cooperative loading, thread communication, global memory, __shared__, extern __shared__

11.3 The 3-Step Recipe

This section explains the three-step shared-memory recipe through a block reduction, including synchronization, cooperative loading, and handling incomplete blocks.

Key terms: 3-Step Recipe, global memory, shared memory, scratchpad memory, software-managed caching, __syncthreads(), barrier, block, cooperative loading, coalescing

11.4 Producers and consumers

This section explains producer-consumer synchronization in shared memory and when staging data there is worthwhile.

Key terms: producer-consumer dependency, producer, consumer, __syncthreads(), read-after-write, write-after-read, divergent branch, kernel boundary, data-reuse test, shared memory

11.5 Applying the 3-Step Recipe

This section applies the shared-memory 3-Step Recipe to several algorithms and connects its data-reuse principle to production GPU techniques.

Key terms: parallel reduction, stride-halving tree, __syncthreads(), barrier, Blelloch, up-sweep, seed step, down-sweep, prefix values, Hillis-Steele

11.6 Bank conflicts

This section identifies bank conflicts as a cause of shared memory slowdowns.

Key terms: shared memory, access pattern, bank conflicts

11.7 Predicting bank conflicts from the access pattern

This section explains how warp membership and shared-memory access patterns determine bank conflicts and their effect on individual accesses.

Key terms: warp, threadIdx.x, threadIdx.y, row-major flattening, shared memory, banks, bank conflict, element index mod 32, stride, N-way bank conflict

11.8 Avoiding bank conflicts with padding

This section explains how an extra shared-memory column prevents column-access bank conflicts and weighs its memory cost against the benefit.

Key terms: padding, row-major order, banks, spacer column, 32-way conflict, shared memory, occupancy

11.9 A first taste of tiling: transpose

This section uses a tiled matrix transpose to demonstrate coalesced global access, synchronization, and conflict-free shared-memory access.

Key terms: shared memory tile, coalesced, __syncthreads(), threadIdx.x, threadIdx.y, bank conflict, padding, tile[32][32], tile[32][33], bounds checks

11.10 Common mistakes

This section identifies shared-memory mistakes that can cause slowdowns, invalid results, or launch failures.

Key terms: warp, bank, shared memory, [32][32] tile, block, occupancy, initialized

11.11 Mistake 1: Bank conflicts in column access

This section explains how padding a shared-memory tile avoids bank conflicts during column access.

Key terms: bank conflicts, shared memory, column access, padding, warp

11.12 Mistake 2: Exceeding shared memory limits

This section explains how shared memory limits can cause kernel launch failures or reduce performance, and how to check and stay within the applicable limit.

Key terms: shared memory per block, cudaGetLastError(), __shared__, prop.sharedMemPerBlock, prop.sharedMemPerBlockOptin, cudaFuncSetAttribute, shared memory, SM, blocks, warps

11.13 Mistake 3: Reading uninitialized shared memory

This section explains why reading uninitialized shared memory causes unreliable results and how to prevent it.

Key terms: __shared__ memory, indeterminate, __syncthreads(), zero-initialized

11.14 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 10: Memory access patterns Chapter 12: Parallel matrix multiplication →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.