liveBook · Manning

Vectorization

As explained in GPU Programming with Triton

Vectorization is an optimization technique that performs one operation on multiple data elements using vector or SIMD instructions.

Vectorization expresses loads, computation, stores, and masking over a block of elements rather than one scalar at a time. A vectorized operation is not necessarily a single hardware instruction. section 2.1.1

Figure 2.1 CUDA assigns one thread to each element, so two blocks of four threads (T0-T3) each compute four scalar additions individually. Triton assigns one program instance to each whole block, so program P0 computes z[0:4] = x[0:4] + y[0:4] and program P1 computes z[4:8] = x[4:8] + y[4:8] as single block-level operations.
CH02 F01 Fartale

The comparison shows how Triton expresses the same additions as block-wide operations instead of assigning each element to an individually managed thread. section 2.1.1

Vector addition

For vectors of eight elements and a block size of four, each Triton program instance computes four additions, such as z[0:4] = x[0:4] + y[0:4]. section 2.1.1

How GPU Programming with Triton covers it

How deep it goes
Explains block-wide vector operations and an eight-element addition, but does not equate them with single hardware instructions.
Examples
Uses two length-eight vectors split into blocks of four to illustrate vector addition.
What it assumes
Assumes vectors or tiled data, while removing the need to coordinate individual GPU threads manually.
What you can do after
Apply it via Triton by expressing a block's loads, computation, stores, and masks as vector operations.
How it connects
Connects vectorization to Triton blocks and tiles, Single Program Multiple Data execution, and masked memory access.

Also explained in

Compare how 22 books explain vectorization →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.