liveBook · Manning

Co-location

As explained in Latency

Colocation is the placement of communicating systems or services in the same physical facility or nearby location to reduce communication distance.

Co-location reduces latency by placing communicating components closer together, so data travels less distance. It applies both across machines, such as an API server and database, and within a machine, where CPU and memory placement affect access latency. section 3.0 section 3.1

Move a database closer

A web application querying a database 6,000 km away might incur 60 ms of network round-trip latency per request. Moving the database to the same data center can reduce query latency to under 10 ms in this example. Embedding the database in the application goes further by eliminating network communication for database access. section 3.0

For a latency-sensitive application, work from distributed placement toward local hardware placement:

  • First examine communication between nodes. Place an API server near its database or another system it frequently contacts; network latency can outweigh application-level optimizations. section 3.1
  • Then examine placement within each machine. On a NUMA system, arrange compute and data so a CPU core accesses memory on its local NUMA node. section 3.1 section 3.6
  • Consider CPU cache behavior and the network stack as additional sources of internal latency. A thread-per-core approach is one way to organize data for CPU access. section 3.1 section 3.6

Geographic proximity does not remove every network delay. Last-mile connections, including Wi-Fi, can still add milliseconds or tens of milliseconds, and their infrastructure may be difficult to control. CDNs and edge computing address distance by moving data or computation closer to users. section 3.2.1 section 3.6

Figure 3.2 Geographic latency. The geographic distance between two nodes implies latency. For example, Hibernia Express advertises 60 ms round-trip between New York and London. Light travels the same distance in vacuum in 19 ms, so the minimum theoretical round-trip latency is 38 ms.
Figure 3.2: Geographic latency. The geographic distance between two nodes implies latency. For example, Hibernia Express advertises 60 ms round-trip between New York and London. Light tarvels the same distance in vacuum in 19 ms so the minimum theoretical round-trip latency is 38 ms.

Compare an advertised New York–London round trip with the theoretical minimum imposed by distance and light travel. section 3.2.1

How Latency covers it

How deep it goes
Explains placement from geographic distance to memory topology, using latency comparisons and a practical database deployment example.
Examples
Compares a remote database with same-data-center placement and embeds SQLite in a REST API server.
What it assumes
Assumes familiarity with latency as a performance metric and with latency modeling and measurement.
What you can do after
Evaluate placement options for latency-sensitive components across machines and within multicore systems.
How it connects
Connects placement to NUMA, CPU caches, network stacks, CDNs, and edge computing.

Also explained in

Compare how 2 books explain co-location →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Latency. The section links point to the parts of the book it draws on.