liveBook · Manning

Tiled matrix multiplication

As explained in Grokking GPU Programming

Tiled matrix multiplication divides matrix operands into smaller blocks that can be computed using local data reuse.

Tiling speeds up GPU matrix multiplication by reusing data. A thread block loads matching chunks of both input matrices into shared memory, then uses those chunks to compute its output tile before moving to the next pair. This reduces repeated trips to slower global memory. section 12.0 section 12.2

Figure 12.6 The big picture of tiling. C is partitioned into T×T tiles, one per thread block; Block(m,n) owns the highlighted tile. To compute it, the block walks tile-row m of A and tile-column n of B, loading one T×T chunk of each into shared memory, computing on the pair, and moving on to the next chunk.

Each block owns one tile of the output matrix and loads the corresponding chunks of the input matrices. section 12.2

For the one-element-per-thread kernel, keep the tile and block dimensions matched:

  • Each thread loads one slot in each shared-memory tile, so the block must have TILE_SIZE × TILE_SIZE threads. section 12.4
  • A smaller block leaves tile slots unfilled; a larger one can overwrite slots or write out of bounds. Tile sizes must also respect thread and shared-memory limits. section 12.4 section 12.13

The shared whiteboard

Think of a thread block as students sharing a whiteboard: they copy values from a distant filing cabinet once, then reuse them there instead of making repeated trips. section 12.2

How Grokking GPU Programming covers it

How deep it goes
Traces block-to-output mapping, cooperative tile loading, and dimension errors, and discusses tile resource limits.
Examples
Uses a student-and-whiteboard analogy and a 32×32 tile to illustrate reuse and thread-to-slot mapping.
What it assumes
Builds on shared memory, previously introduced through a transpose kernel.
What you can do after
Build a CUDA tiled matrix-multiplication kernel by mapping blocks to output tiles and loading shared-memory tiles cooperatively.
How it connects
Links matrix multiplication to CUDA thread blocks, shared memory, global memory, and cuBLAS.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.