liveBook · Manning

Tile

A tile is a bounded block of elements from a larger array, matrix, or tensor that is treated as a unit of computation.

How 3 Manning books explain it

GPU Programming with Triton

A tile is a portion of a tensor processed by one Triton program instance.

How it is used
Organizes tensor work into blocks processed by parallel Triton program instances.
How deep it goes
Develops tile addressing, masking, data reuse, and a tiled matrix-multiplication kernel, initially without boundary or remainder handling.
Examples
Uses vector addition and a 4 × 4 matrix product distributed across four tiled programs.
What you can do after
Apply it via Triton to address tensor blocks and construct a tiled matrix-multiplication kernel.

Read how GPU Programming with Triton explains tile →

CUDA for LLMs

A tile is a bounded subset of an array or matrix processed as a unit.

How it is used
Uses tiles to reduce memory traffic and distribute matrix multiplication and attention work across GPU execution units.
How deep it goes
Traces tile shapes, memory loads, accumulation, register reuse, and synchronization through CUDA kernels and a producer-consumer pipeline.
Examples
Works through a 128 × 128 GEMM output tile, 8 × 8 thread micro-tiles, and tiled Flash Attention.
What you can do after
Build a tiled CUDA matrix multiplication using shared-memory input tiles and register accumulation.

Read how CUDA for LLMs explains tile →

Grokking GPU Programming

A tile is a smaller rectangular block of elements taken from a larger matrix or other data structure.

How it is used
Uses tiles to stage matrix chunks in shared memory for reuse and efficient global-memory access.
How deep it goes
Explains tile ownership, partial sums, thread-to-slot mapping, synchronization, and padding, including consequences of mismatched block dimensions.
Examples
Uses a filing-cabinet and whiteboard analogy, tiled matrix multiplication, and a padded transpose kernel.
What you can do after
Diagnoses incorrect tile loading caused by mismatched thread-block and tile dimensions.

Read how Grokking GPU Programming explains tile →

Also covered in

What to know first

Related concepts

Questions about tile

What is tile?

A tile is a bounded block of elements from a larger array, matrix, or tensor that is treated as a unit of computation.

Which Manning books explain tile?

GPU Programming with Triton, CUDA for LLMs and Grokking GPU Programming.

What should I know before learning tile?

Manning books that teach tile usually introduce matrix-matrix multiplication, float16, hardware register and inner product first.

What is tile part of?

Tile is part of matrix. A matrix is a rectangular two-dimensional array of elements arranged in rows and columns.

What are the parts of tile?

Tile includes WMMA fragment and micro-tile.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.