Repeated global-memory reads can leave GPU threads waiting for data. L1 cache is a small, hardware-managed cache private to each streaming multiprocessor (SM). It holds recently accessed global-memory data so later accesses can hit nearby storage instead of reaching DRAM. section 1.4 section 3.2.3
L1 is part of a hierarchy that also includes registers, shared memory, L2 cache, and global memory. L2 is shared across the GPU, while each SM has its own L1. An L1 hit is generally much faster than an access to L2 or high-bandwidth memory (HBM). section 3.2 section 3.2.3
Figure 1.8 The memory hierarchy organized by speed and size. Registers top the hierarchy as the fastest and smallest storage; global memory sits at the bottom as the largest and slowest. Shared memory and the L1 cache appear on adjacent rows to reflect their distinct programming roles; on NVIDIA GPUs from the Volta architecture onwards, they share the same physical on-chip SRAM on each SM and have the same access latency, so the figure separates them by function, not by speed.
The hierarchy separates L1 and shared memory by programming role, even where they share physical storage and access latency. section 1.4
On NVIDIA GPUs from Volta onward, L1 and shared memory use the same on-chip SRAM and have the same access latency, but serve different roles. Hardware manages L1 automatically; shared memory supports explicit cooperation and data placement. Partitioning the SRAM gives a kernel a trade-off: more shared memory leaves less L1, and vice versa. section 1.4 section 3.2.2 section 3.2.3
H100 storage trade-off
The NVIDIA H100 provides 256 KB of unified L1 and shared-memory storage with different partitioning choices. A kernel that reuses explicitly placed data may benefit from more shared memory; one that benefits more from hardware-managed caching may prefer more L1. In Triton, programmers do not declare shared-memory buffers explicitly: the compiler and backend may use on-chip storage when lowering blocked tensor operations. section 3.2.2

