Tile shape
A tile shape specifies the dimensions or dimensional arrangement of the elements in a tile.
CUDA for LLMs
A thread block tile shape specifies the dimensions of a computation tile assigned to a GPU thread block.
- How it is used
- Configures GEMM work per thread block and exposes a key performance-tuning choice in CUTLASS.
- How deep it goes
- Explains output and inner-tile dimensions, resource tradeoffs, and empirical tuning without prescribing a universally optimal shape.
- Examples
- Uses Shape<_128, _256, _64> for a GEMM computing 128 × 256 output tiles from smaller A and B tiles.
- What you can do after
- Configure and tune CUTLASS GEMM tile shapes using the profiler, while checking resource use and alignment compatibility.
GPU Programming with Triton
Tile shape specifies the dimensional arrangement of elements in a tile.
- How it is used
- Configures how Triton partitions matrix operands and outputs into blocks for a tiled matrix-multiplication kernel.
- How deep it goes
- Defines BLOCK_M, BLOCK_N, and BLOCK_K through pointer tiles, register accumulators, K-loop updates, and output writes; the first kernel assumes divisible dimensions.
- Examples
- Compares square and rectangular output tiles and traces a 4 × 4 product across four programs.
- What you can do after
- Build it from scratch by assigning output tiles to programs and accumulating successive K-slices with tl.dot.
What to know first
- hardware registerA hardware register is a small, rapidly accessible storage location within a processor or processing device that holds data, addresses, or control information during execution.
- matrix-matrix multiplicationMatrix multiplication is an operation on compatible matrices in which each entry of the product is the dot product of a row of the first matrix and a column of the second.
- shared memoryShared memory is a memory region accessible to multiple processes, threads, or other concurrent execution contexts for sharing data.
- tileA tile is a bounded block of elements from a larger array, matrix, or tensor that is treated as a unit of computation.
Related concepts
- arithmetic intensityconfigures · Arithmetic intensity is the ratio of computational operations performed to bytes of data transferred by a computation.
- autotuningused by · Autotuning automatically evaluates and selects or adjusts algorithm, implementation, or configuration choices to improve a measured objective, such as performance.
- tile sizeconfigured by · Tile size specifies the dimensions or number of elements in a tile, a portion of data processed as a unit.
Questions about tile shape
What is tile shape?
A tile shape specifies the dimensions or dimensional arrangement of the elements in a tile.
Which Manning books explain tile shape?
CUDA for LLMs and GPU Programming with Triton.
What should I know before learning tile shape?
Manning books that teach tile shape usually introduce hardware register, matrix-matrix multiplication, shared memory and tile first.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

