liveBook · Manning

Grid

As explained in GPU Programming with Triton

A GPU grid is the complete collection of blocks or program instances launched for one kernel call.

A GPU grid is the complete collection of blocks launched for one kernel call. In Triton, each grid position corresponds to a program instance: the kernel describes the work for one tile, and the grid scales that work across the full input. section 1.3.3 section 2.2

The grid specifies the total work, not how many instances run simultaneously. The GPU schedules blocks across streaming multiprocessors, which run blocks as resources permit and take more work until the grid is complete. Triton handles thread-level execution within each block. section 1.3.3 section 2.4

Figure 1.7 The grid contains every block launched for the kernel, and the GPU schedules those blocks across its Streaming Multiprocessors. Each SM runs one or more blocks at a time depending on available resources and continues pulling new blocks until the grid is fully processed.
Block scheduling on GPU Streaming Multiprocessors

The grid contains all launched blocks; streaming multiprocessors execute them as resources become available. section 1.3.3

For a vector of N elements and a block size B, a one-dimensional grid needs ceil(N / B) instances. The host can compute this with triton.cdiv and pass the resulting tuple to the kernel launch. A grid function can read the resolved BLOCK_SIZE compile-time constant from its meta argument, so the grid adapts to the chosen block size. section 2.2.8

grid = lambda meta: (
    triton.cdiv(n_elements, meta['BLOCK_SIZE']),
)

This grid function computes the number of instances along axis 0 from the input length and resolved block size. section 2.2.8

Each instance uses tl.program_id(0) to identify its position and computes its starting offset as program ID × BLOCK_SIZE. It adds local offsets to find its elements. If the vector length is not a multiple of the block size, a mask prevents the last instance from loading or storing out-of-range positions. section 2.2.3 section 2.2.9

Vector addition

For 98,432 elements and a block size of 1,024, the grid has 97 instances. Each instance adds a distinct slice of the two input vectors and stores the matching output slice; the last instance masks its unused positions. section 2.2.9 section 2.2

A Triton launch grid can have up to three dimensions. For example, a two-dimensional grid maps program IDs to row and column tiles of a matrix; boundary masks protect tiles that extend beyond its dimensions. section 2.2.3 section 5.10

How GPU Programming with Triton covers it

How deep it goes
Explains grid sizing, program IDs, boundary masking, launch metadata, and scheduling across streaming multiprocessors.
Examples
Uses vector addition and a two-dimensional matrix grid to illustrate work assignment.
What you can do after
Apply it via Triton to size a launch grid and map program instances to data tiles.
How it connects
Connects grids to kernels, blocks, tiles, program IDs, masking, and streaming multiprocessors.

Also explained in

Compare how 3 books explain grid →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.