liveBook · Manning

DRAM

As explained in GPU Programming with Triton

DRAM is a capacitor-based memory technology that requires periodic refresh and provides high storage density at higher access latency than SRAM.

DRAM stores each bit as charge on a capacitor, which must be refreshed periodically. Its one-transistor-per-bit cells provide more capacity per area than SRAM, at the cost of higher access latency. GPUs use DRAM for global memory, where large tensor and model storage matters more than the low latency of on-chip memory. section 1.4

Figure 1.8 The memory hierarchy organized by speed and size. Registers top the hierarchy as the fastest and smallest storage; global memory sits at the bottom as the largest and slowest. Shared memory and the L1 cache appear on adjacent rows to reflect their distinct programming roles; on NVIDIA GPUs from the Volta architecture onwards, they share the same physical on-chip SRAM on each SM and have the same access latency, so the figure separates them by function, not by speed.
CH01 F07 Fartale

See where DRAM-based global memory sits in the GPU memory hierarchy relative to caches and registers. section 1.4

How GPU Programming with Triton covers it

How deep it goes
Defines capacitor-based storage and periodic refresh, then contrasts DRAM’s density and latency with SRAM.
Examples
Uses GPU global memory, often HBM, as the DRAM-based storage example.
What you can do after
Understand why global-memory accesses cost far more than accesses to on-chip storage.
How it connects
Connects DRAM to SRAM, global memory, and the GPU memory hierarchy.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.