GPU video memory, often called global memory or VRAM, holds the bulk of a GPU’s data: inputs, outputs, and persistent state. It is large, device-wide DRAM, but accessing it takes much longer than accessing on-chip registers. A discrete GPU has dedicated VRAM; an integrated GPU may share memory with the CPU. section 3.2.4 section 3.6.1
Global memory sits at the bottom of a hierarchy that also includes registers, shared memory, L1 cache, and L2 cache. A request that reaches DRAM travels from a streaming multiprocessor through L2 and a memory controller, then returns along the path. The round trip can take hundreds of clock cycles. High bandwidth does not eliminate that latency: many aligned requests can be in flight at once, while other threads compute to hide the wait. section 1.4 section 3.2.4
Figure 1.8 The memory hierarchy organized by speed and size. Registers top the hierarchy as the fastest and smallest storage; global memory sits at the bottom as the largest and slowest. Shared memory and the L1 cache appear on adjacent rows to reflect their distinct programming roles; on NVIDIA GPUs from the Volta architecture onwards, they share the same physical on-chip SRAM on each SM and have the same access latency, so the figure separates them by function, not by speed.
Compare global memory’s capacity and speed with the on-chip levels that reduce trips to it. section 1.4
In Triton, a program forms addresses by adding element offsets to a base pointer. tl.load reads a block of values from global memory into registers for computation; tl.store writes results back. A mask prevents loads and stores from touching positions beyond the array boundary when a tile is only partially filled. section 2.2.7
Avoiding an intermediate memory round trip
For z = x * sigmoid(x) * y, separate SiLU and multiplication kernels write an intermediate to global memory and read it back. With float32 values, that costs five transfers, or 20 bytes per element. A fused kernel loads x and y, computes in registers, and stores z: three transfers, or 12 bytes per element. The arithmetic is mathematically equivalent, apart from possible minor floating-point differences. section 3.3
Reducing global-memory traffic matters because compute throughput can outpace the bandwidth that supplies it, leaving arithmetic units idle: the memory wall. Fusion raises arithmetic intensity by doing more work per byte moved, though register use must still be balanced against the occupancy needed to hide memory latency. section 3.1 section 3.7





