liveBook · Manning

Tiling

Tiling is an optimization technique that partitions data or computation into smaller blocks to improve locality, reuse, and memory efficiency.

How 3 Manning books explain it

CUDA for LLMs

Tiling is a computation technique that partitions data into smaller blocks to improve locality and reuse.

How it is used
Optimizes GPU matrix multiplication and attention by keeping reusable tiles and intermediates near computation.
How deep it goes
Traces tiled GEMM from shared-memory scheduling and synchronization through register micro-tiles and warp-level WMMA fragments.
Examples
Uses 128 × 128 GEMM block tiles, 8 × 8 thread micro-tiles, and tiled Flash Attention.
What you can do after
Build tiled CUDA kernels using shared-memory staging, synchronization, and register accumulation.

Read how CUDA for LLMs explains tiling →

Grokking GPU Programming

Tiling is a technique that partitions data or computation into smaller blocks to enable reuse and reduce memory traffic.

How it is used
Reduces repeated global-memory requests by staging matrix chunks for cooperative reuse in shared memory.
How deep it goes
Works through tiled matrix multiplication, including block mapping, boundary checks, synchronization, traffic counts, and tile-size limits.
Examples
Uses a study-group whiteboard analogy and compares naive and 32×32-tiled multiplication of 1024×1024 and 1024×512 matrices.
What you can do after
Builds a tiled matrix multiplication kernel from scratch using cooperative loads, synchronization, accumulation, and boundary checks.

Read how Grokking GPU Programming explains tiling →

Transformers in Action

Tiling is a technique that partitions a large computation into smaller subproblems whose data can be reused in faster local memory.

How it is used
Uses tiling to reduce GPU memory traffic in matrix multiplication and attention.
How deep it goes
Explains cooperative tile loading, blockwise softmax, and hardware overlap without developing a kernel from scratch.
Examples
Illustrates tiled matrix multiplication and blockwise query-key-value attention.
What you can do after
Apply it via FlashAttention-3 by selecting its attention implementation when loading a causal language model.

Read how Transformers in Action explains tiling →

Also covered in

What to know first

Related concepts

Questions about tiling

What is tiling?

Tiling is an optimization technique that partitions data or computation into smaller blocks to improve locality, reuse, and memory efficiency.

Which Manning books explain tiling?

CUDA for LLMs, Grokking GPU Programming and Transformers in Action.

What should I know before learning tiling?

Manning books that teach tiling usually introduce attention, matrix-matrix multiplication, arithmetic intensity and attention matrix first.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.