Co-location reduces latency by placing communicating components closer together, so data travels less distance. It applies both across machines, such as an API server and database, and within a machine, where CPU and memory placement affect access latency. section 3.0 section 3.1
Move a database closer
A web application querying a database 6,000 km away might incur 60 ms of network round-trip latency per request. Moving the database to the same data center can reduce query latency to under 10 ms in this example. Embedding the database in the application goes further by eliminating network communication for database access. section 3.0
For a latency-sensitive application, work from distributed placement toward local hardware placement:
- First examine communication between nodes. Place an API server near its database or another system it frequently contacts; network latency can outweigh application-level optimizations. section 3.1
- Then examine placement within each machine. On a NUMA system, arrange compute and data so a CPU core accesses memory on its local NUMA node. section 3.1 section 3.6
- Consider CPU cache behavior and the network stack as additional sources of internal latency. A thread-per-core approach is one way to organize data for CPU access. section 3.1 section 3.6
Geographic proximity does not remove every network delay. Last-mile connections, including Wi-Fi, can still add milliseconds or tens of milliseconds, and their infrastructure may be difficult to control. CDNs and edge computing address distance by moving data or computation closer to users. section 3.2.1 section 3.6
Figure 3.2 Geographic latency. The geographic distance between two nodes implies latency. For example, Hibernia Express advertises 60 ms round-trip between New York and London. Light travels the same distance in vacuum in 19 ms, so the minimum theoretical round-trip latency is 38 ms.
Compare an advertised New York–London round trip with the theoretical minimum imposed by distance and light travel. section 3.2.1

