liveBook · Manning

Memory bank

As explained in Grokking GPU Programming

A memory bank is an independently accessed division of memory that can serve requests concurrently with other banks.

A memory bank is an independently accessed part of memory. Shared memory uses banks as separate pathways: threads in one warp can access different banks at the same time, but requests for different data in the same bank serialize. section 11.7

For shared-memory accesses, the pattern matters:

  • With stride-1 access, each of 32 warp threads reaches a different bank, so the warp is served in one transaction. section 11.7
  • With stride-2 access, threads share 16 even-numbered banks. The warp needs two rounds, while the odd-numbered banks sit idle. section 11.7
  • Padding array indices can break a regular conflicting stride. Blelloch scan uses this idea, though its example leaves padding unimplemented; production libraries such as CUB handle it automatically. section 7.11

A repeating bank pattern

In the described 32-bank model, an element’s bank is its index modulo 32: elements 0 and 32 both map to bank 0. A warp accessing different addresses that map to one bank encounters a conflict. section 11.7

Figure 11.5 Shared memory is divided into 32 banks, and array elements are assigned to them round-robin: an element’s bank is its index mod 32. The assignment wraps every 32 elements: element 0 and element 32 both land in bank 0, element 1 and element 33 both land in bank 1, and so on.

The diagram shows how array elements map round-robin across banks, with the pattern repeating every 32 elements. section 11.7

How Grokking GPU Programming covers it

How deep it goes
Derives bank conflicts from access patterns and gives a padding remedy, while deferring wider-access and other-architecture mappings.
Examples
Uses stride-1 and stride-2 warp accesses, plus Blelloch scan padding, to illustrate conflicts and their mitigation.
What you can do after
Diagnose shared-memory slowdowns by mapping warp accesses to banks and recognizing when padding can reduce conflicts.
How it connects
Connects shared memory to warps, array indexing, and CUDA scan and matrix-tile access patterns.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.