liveBook · Manning

Tile shape

A tile shape specifies the dimensions or dimensional arrangement of the elements in a tile.

How 2 Manning books explain it

CUDA for LLMs

A thread block tile shape specifies the dimensions of a computation tile assigned to a GPU thread block.

How it is used
Configures GEMM work per thread block and exposes a key performance-tuning choice in CUTLASS.
How deep it goes
Explains output and inner-tile dimensions, resource tradeoffs, and empirical tuning without prescribing a universally optimal shape.
Examples
Uses Shape<_128, _256, _64> for a GEMM computing 128 × 256 output tiles from smaller A and B tiles.
What you can do after
Configure and tune CUTLASS GEMM tile shapes using the profiler, while checking resource use and alignment compatibility.

Read how CUDA for LLMs explains tile shape →

GPU Programming with Triton

Tile shape specifies the dimensional arrangement of elements in a tile.

How it is used
Configures how Triton partitions matrix operands and outputs into blocks for a tiled matrix-multiplication kernel.
How deep it goes
Defines BLOCK_M, BLOCK_N, and BLOCK_K through pointer tiles, register accumulators, K-loop updates, and output writes; the first kernel assumes divisible dimensions.
Examples
Compares square and rectangular output tiles and traces a 4 × 4 product across four programs.
What you can do after
Build it from scratch by assigning output tiles to programs and accumulating successive K-slices with tl.dot.

Read how GPU Programming with Triton explains tile shape →

What to know first

Related concepts

Questions about tile shape

What is tile shape?

A tile shape specifies the dimensions or dimensional arrangement of the elements in a tile.

Which Manning books explain tile shape?

CUDA for LLMs and GPU Programming with Triton.

What should I know before learning tile shape?

Manning books that teach tile shape usually introduce hardware register, matrix-matrix multiplication, shared memory and tile first.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.