liveBook · Manning

Mask

As explained in GPU Programming with Triton

A mask is a per-element boolean predicate that enables or suppresses selected elements of a GPU memory operation.

A mask is a per-element boolean predicate that controls which positions of a GPU memory operation are active. It is especially useful when a fixed-size block extends beyond the end of an array: valid positions are accessed, while invalid positions are skipped without branching. section 2.2.6 section 2.4

Vector addition at the boundary

Each Triton program computes a vector of offsets for its block. The expression `mask = offsets < n_elements` marks the offsets inside the array. Pass that mask to both `tl.load` for the inputs and `tl.store` for the output. The last program can then process a partially filled block without accessing memory beyond the vectors. section 2.2.6 section 2.2.7

A masked load can supply an `other` value for inactive positions; a masked store simply skips them. Choose the load value for the computation that follows, not just for memory safety. Without `other`, a masked load’s value is undefined and can affect valid results if it enters a computation such as `tl.dot`. section 5.3.3 section 5.8.1

Different computations need different replacement values:

  • For an incomplete matrix-multiplication tile, use `other=0.0` on input loads so extra products contribute nothing to the accumulator. section 5.8.1
  • For a fused softmax, use `other=-inf`: masked positions do not change the maximum and become zero after exponentiation, so they do not contribute to the sum. section 4.2.3

For a two-dimensional tile, a position is valid only if both its row and column are in range. Expand the row and column offset vectors along opposite axes, then combine their comparisons with elementwise AND. Broadcasting produces a boolean mask with the shape of the tile. section 5.3.3

mask = (tl.expand_dims(row_offs, 1) < M) & \
       (tl.expand_dims(col_offs, 0) < N)

This expression combines row and column bounds into a mask for a matrix tile. section 5.3.3

A partially valid matrix tile

For a 35×70 matrix tiled in 32×32 blocks, the tile starting at row 32 and column 64 extends past both boundaries. Its mask enables only rows 32–34 and columns 64–69; loads and stores skip the remaining positions. section 5.3.3

Boolean masks also drive per-lane value selection with `tl.where`, but selection is not a substitute for masking a memory operation. `tl.where` selects between candidate values after their computation, whereas a masked load prevents the inactive memory access itself. section 3.4.2

How GPU Programming with Triton covers it

How deep it goes
Explains predicate construction, masked load and store behavior, multidimensional broadcasting, and replacement values for subsequent computations.
Examples
Works through a partially covered matrix tile and a fused softmax with negative-infinity replacement.
What it assumes
Assumes intermediate Python and introduces masking alongside a first Triton vector-addition kernel.
What you can do after
Apply it via Triton to guard boundary accesses and select replacement values appropriate to the computation.
How it connects
Connects masking to tiling, predicated execution, reductions, and matrix multiplication.

Also explained in

Compare how 9 books explain mask →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.