liveBook · Manning

Padding

As explained in Grokking GPU Programming

Padding adds extra elements to a data structure, commonly with neutral values, to meet an algorithm's size requirements.

When threads read down the columns of a shared-memory tile, their addresses can all map to the same bank, forcing accesses to run in sequence. Padding adds an unused spacer column to shift each row’s starting address and spread those accesses across banks. section 11.8 section 11.11

Figure 11.8 The padding trick on a toy 4-bank GPU: without padding (arr[4][4]), row-major storage places every column-0 element at a multiple-of-4 address, so all four land in bank 0 (a 4-way conflict). Adding one padding column (arr[4][5]) shifts each row’s start by one, moving the four accesses into four different banks, so the conflict disappears.

The toy four-bank example shows how changing arr[4][4] to arr[4][5] spreads column-zero accesses across banks. section 11.8

A spacer column

For a 32 × 32 tile, changing tile[32][32] to tile[32][33] spreads a column’s accesses across 32 banks. The extra column costs 3.125% more shared memory; threads still use only columns 0–31. section 11.8

Padding can also mean extending a Blelloch scan’s input to the next power of two with the identity value for addition, zero. Ignore the added positions afterward. For scan bank conflicts, padding shared-memory indices can help, but profile before implementing it; libraries such as CUB handle this automatically. section 7.12 section 7.11

How Grokking GPU Programming covers it

How deep it goes
Shows how a spacer column shifts bank alignment, quantifies its memory cost, and cautions that other element sizes may need different mappings.
Examples
Uses a four-bank 4 × 4 array, a 32-bank 32 × 32 tile, and a six-element scan padded with zeros.
What it assumes
Explains row-major indexing and bank mapping with a toy model; scan padding relies on knowing the identity element for addition.
What you can do after
Apply it by adding a spacer column to a column-accessed tile or identity values to a power-of-two scan input.
How it connects
Connects shared memory and bank conflicts with tiled arrays, and connects identity padding with Blelloch scans.

Also explained in

Compare how 22 books explain padding →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.