Non-uniform memory access (NUMA) is a multicore memory topology in which access latency depends on the relationship between a CPU and a memory location. Memory attached to the CPU’s local NUMA node is faster to access than memory attached to a remote node. Uniform memory access (UMA) instead gives cores consistent main-memory latency. section 3.4
NUMA makes data placement part of application performance. Keeping a working set near the CPU that processes it reduces latency, while repeatedly accessing remote memory imposes a cost on each access. Cache behavior matters too: L1 access can be up to 100 times faster than DRAM access, so large or poorly accessed working sets can compound NUMA penalties. section 3.4 section 7.4.5
A practical NUMA-aware memory strategy typically includes:
- Discover the machine’s memory topology with the Portable Hardware Locality (hwloc) library or operating-system services. section 7.4.5
- Partition application state across NUMA nodes or CPU cores so requests execute near the data they use. section 3.4
- Bind selected data structures to appropriate CPUs or memory partitions with operating-system facilities such as Linux mbind(). section 7.4.5
- Process large datasets in cache-sized batches to reduce cache thrashing and improve locality. section 7.4.5
Partitioning a key–value store
A key–value store can divide its keyspace across NUMA nodes or CPU cores. When a lookup arrives, the server forwards it to the partition responsible for that key, keeping processing close to the partition’s memory. section 3.4
Figure 3.4 UMA and NUMA architectures. The left side of the figure represents a UMA architecture where all CPU cores have the same access latency to DRAM. The right side represents a NUMA architecture with two nodes. CPUs that are in the same node can access NUMA node local memory faster than memory in a remote node.
Compare UMA’s uniform DRAM latency with NUMA’s faster local-node and slower remote-node access. section 3.4
