liveBook · Manning

Memory transaction

As explained in Grokking GPU Programming

A memory transaction is a unit of memory service that processes one or more requests between a processor and memory.

A memory transaction is a unit of memory service. On a GPU, a warp issues a memory instruction together, and the hardware tries to serve its threads’ requests in as few transactions as possible. Fewer transactions mean fewer fetched bytes go unused. section 10.4

For 4-byte floats, the 128-byte cache-line teaching model makes the difference clear:

  • When 32 warp threads request consecutive floats, one transaction can serve them all. section 10.4
  • When threads request unrelated locations, the warp may need up to 32 transactions to fetch 32 floats. section 10.7

One delivery truck or many

Think of each transaction as a truck carrying a 32-slot container. If 32 threads request nearby data, one truck can serve them; if their requests are scattered, separate trips fetch mostly unused space. section 10.4

Figure 10.5 Scattered access: 32 threads, up to 32 transactions. The warp’s 32 threads (T0 … T31) each read from an unrelated location in memory, so their addresses fall into up to 32 different 128-byte cache lines (the brackets at indices 32*i, 32*j, 32*k, …). Because no two threads share a cache line, the hardware must issue a separate transaction for each one, and only 4 bytes out of every 128-byte transaction are actually used, roughly 3% bandwidth efficiency in this chapter’s cache-line model (about 12.5% counted in 32-byte sectors), the worst case in this chapter. Following the Snapshot Rule, picture 32 arrows fanning out to 32 different cache lines, each one carrying a single useful element.

The figure shows how scattered warp requests can fan out across separate cache lines, wasting most of each transaction. section 10.7

How Grokking GPU Programming covers it

How deep it goes
Defines a 128-byte cache-line teaching model, contrasts access patterns, and notes modern GPUs count 32-byte sectors.
Examples
Compares consecutive float loads with scattered warp requests using a delivery-truck analogy.
What it assumes
Assumes familiarity with 32-thread warps and float-sized elements when explaining transaction counts.
What you can do after
Evaluate access patterns by estimating transaction counts and identifying wasted bandwidth.
How it connects
Connects memory transactions to warps, SIMT, coalescing, cache lines, and memory bandwidth.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking GPU Programming. The section links point to the parts of the book it draws on.