A single-process cache can become ineffective when request volume or cached data exceeds one machine’s practical capacity. Distributed caching spreads entries across standalone cache nodes and routes each request to the node responsible for its key. section 7.8 section 7.3
The design separates responsibilities:
- Nodes store cache entries independently. section 7.8
- An orchestrator routes requests to the appropriate node. section 7.8
- Consistent hashing maps keys and nodes into the same identifier space, assigning each key to the next node. section 7.8
- Adding or removing a node remaps only a fraction of keys instead of forcing a complete redistribution. section 7.8
Scaling cached database results
At roughly a million database accesses per second, a cache limited to one process may provide too little protection. Splitting entries across nodes increases capacity, while consistent hashing limits disruption when the cluster changes. section 7.8
Figure 7.14 An array of cache nodes using consistent hashing for distributing keys. On the left, the initial nodes array. Nodes and keys map to the same metric space (the IDs space), usually represented as a circle (which could be obtained even with just a modulo operation on the hash values). Keys are assigned to the next node in the circle—that is, the node whose value is the smallest node’s ID larger than the key’s hashing. On the right, we can see how, when a cache node is removed or added, only a fraction of the keys needs to be remapped.
Follow the key-and-node ring to see how consistent hashing assigns entries and preserves most assignments after a node is added or removed. section 7.8
