liveBook · Manning

Chapter 12: Parallel matrix multiplication

What you will learn in chapter 12 of Grokking GPU Programming

This introduction explains the data bottleneck in naive matrix multiplication and how tiling with shared memory can reduce repeated global-memory access.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This introduction explains the data bottleneck in naive matrix multiplication and how tiling with shared memory can reduce repeated global-memory access.

Key terms: global memory, shared memory, tiling

12.1 Understanding the problem

This section explains naive parallel matrix multiplication and analyzes why redundant memory requests make it memory-bound.

Key terms: dot product, multiply-add, floating-point operations, naive kernel, 2D grid, thread blocks, ceiling division, global memory reads, redundant traffic, unique data

12.2 The solution: tiled matrix multiplication

This section explains how thread blocks tile matrix multiplication and reuse input data through shared memory.

Key terms: tiling, thread block, tile, shared memory, global memory, global memory transaction, data reuse

12.3 Tiles: breaking a matrix into bite-sized chunks

This section explains how matrix tiling enables shared-memory reuse and partial-sum computation, including tile-size limits and floating-point caveats.

Key terms: tile, shared memory, data reuse, tile size, TILE_SIZE, threads per block, dot product, partial sums, tile iteration, matching strips

12.4 Walkthrough of a tiled kernel

This section introduces the tiled matrix multiplication kernel’s block layout, shared-memory tiles, and per-thread accumulation across dot-product chunks.

Key terms: TILE_SIZE, blockDim, threadIdx, shared memory, tile, global memory, data reuse, dot product, sum, tile loop

12.5 Step 1: Load tiles into shared memory

This section explains cooperative loading of matrix tiles into shared memory, including tile indexing and boundary handling.

Key terms: cooperative loading, shared tile arrays, t * TILE_SIZE, aCol, threadIdx.x, threadIdx.y, tileA, boundary check, zero-padding, partial tiles

12.6 Step 2: Synchronize

This section explains why a block-wide barrier is needed between loading shared-memory tiles and computing with them.

Key terms: __syncthreads(), barrier, block, tile

12.7 Step 3: Compute using shared memory

This section explains how threads compute partial dot products from shared-memory tiles without bank conflicts.

Key terms: partial dot product, shared memory, global memory, latency, bank conflict, broadcast, warp, threadIdx.y, threadIdx.x

12.8 Step 4: Synchronize again, then repeat

Completes the tiled matrix multiplication loop and explains its synchronization, memory traffic, arithmetic intensity, and tile-size limits.

Key terms: __syncthreads(), sum, boundary check, dot product, TILE_SIZE, global memory traffic, shared memory, data reuse, arithmetic intensity, FLOPs

12.9 In production: use cuBLAS

This section explains why cuBLAS is the production default for matrix multiplication, how to use it with row-major matrices, and when alternatives or custom kernels are appropriate.

Key terms: cuBLAS, tiling, Tensor Cores, FP32, FP16, cublasSgemm, single-precision, α, β, column-major

12.10 Common mistakes

This section identifies two silent errors that can produce an incorrect matrix product: a missing barrier and a mismatch in matrix indexing.

Key terms: barrier, tile, block, column-major arithmetic, row-major data

12.11 Mistake 1: Missing the second __syncthreads()

This section explains why tiled matrix multiplication needs a second synchronization barrier and why omitting it can be hard to detect.

Key terms: __syncthreads(), shared memory, tileA, tileB, write-after-read race, K, TILE_SIZE, non-deterministic

12.12 Mistake 2: Row-major vs column-major confusion

This section explains how mismatched matrix storage and indexing produce structured errors and how to correct and check them.

Key terms: row-major order, column-major arithmetic, transpose, test case

12.13 Mistake 3: Tile size over the limits

This section checks tile sizes against GPU limits and recaps the inputs, outputs, and cooperative data reuse of tiled matrix multiplication.

Key terms: TILE_SIZE, threads per block, shared memory budget, dot product, row, column, cooperative tile loading, shared tiles, global memory, reuse

12.14 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 11: Shared memory Chapter 13: Read-only data optimization →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.