liveBook · Manning

Tile

As explained in Grokking GPU Programming

A tile is a smaller rectangular block of elements taken from a larger matrix or other data structure.

When a matrix is too large to work on efficiently all at once, divide it into smaller rectangular chunks called tiles. A square tile with side length 32 contains 32 × 32 elements. Working on one tile at a time lets threads reuse nearby data before returning to larger, slower global memory. section 12.3

Think of a group copying values from a distant filing cabinet onto a shared whiteboard. Everyone can reuse the copied values without making another trip. In a GPU kernel, the cabinet is global memory, the whiteboard is shared memory, and the group is a thread block. section 12.2

Matrix multiplication, one output tile at a time

For C = A × B, each thread block owns one square tile of C. It loads matching tiles from a row of A and a column of B into shared memory, computes partial results, and repeats with the next pair. Accumulating those partial results produces the output tile without loading the whole matrices at once. section 12.2 section 12.3

Figure 12.6 The big picture of tiling. C is partitioned into T×T tiles, one per thread block; Block(m,n) owns the highlighted tile. To compute it, the block walks tile-row m of A and tile-column n of B, loading one T×T chunk of each into shared memory, computing on the pair, and moving on to the next chunk.

Follow one output tile as its thread block moves through matching tiles of A and B. section 12.2

In the one-element-per-thread multiplication kernel, the block dimensions must match the tile dimensions. A 16 × 16 block loading a 32 × 32 tile leaves three-quarters of its slots unfilled; a block larger than its tile can overwrite slots or write outside the shared arrays. Other loading schemes are possible, but they need additional logic. section 12.4

Transpose with a padded tile

A transpose reads rows and writes columns. A 32 × 32 shared-memory tile lets threads read and write consecutive global addresses while rearranging the values inside shared memory. A barrier separates loading from transposed reading. Padding the tile to 32 × 33 avoids the 32-way shared-memory bank conflict of a 32 × 32 array. The example assumes matrix dimensions are multiples of 32; other sizes need bounds checks. section 11.9

__global__ void transposeTiled(const float* in, float* out, int n) {
    __shared__ float tile[32][33];    #A
    int x = blockIdx.x * 32 + threadIdx.x;
    int y = blockIdx.y * 32 + threadIdx.y;
    tile[threadIdx.y][threadIdx.x] = in[y * n + x];    #B
    __syncthreads();    #C
    x = blockIdx.y * 32 + threadIdx.x;
    y = blockIdx.x * 32 + threadIdx.y;
    out[y * n + x] = tile[threadIdx.x][threadIdx.y];    #D
}

See the padded tile, barrier, and swapped indices in the transpose kernel. section 11.9

How Grokking GPU Programming covers it

How deep it goes
Explains tile ownership, partial sums, thread-to-slot mapping, synchronization, and padding, including consequences of mismatched block dimensions.
Examples
Uses a filing-cabinet and whiteboard analogy, tiled matrix multiplication, and a padded transpose kernel.
What you can do after
Diagnoses incorrect tile loading caused by mismatched thread-block and tile dimensions.
How it connects
Connects tiling to matrix multiplication, matrix transpose, shared memory, loop blocking, and distributed-memory algorithms.

Also explained in

Compare how 3 books explain tile →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.