liveBook · Manning

Latency

As explained in Grokking Software Architecture

Latency is the elapsed time between a request or input and the corresponding response or observable result.

A customer clicks “Place order” and watches a spinner. Latency is the elapsed time between a request or input and its response or observable result. It is one aspect of performance; throughput, by contrast, measures how much work gets done per unit of time. section 2.4

Network calls add travel time, and waiting for them one after another adds their delays to the user’s wait. Treating network latency as zero is a dangerous assumption in distributed systems. In-memory calls avoid that network trip, though architecture choices involve other tradeoffs too. section 12.6.1 section 12.1

The order that waits for everything

An Order API checks inventory, charges a card, and prints a shipping label in sequence. If each API takes 200 milliseconds, the customer waits 600 milliseconds, plus network time. There’s an availability trap, too: if Shipping is down, the order cannot be placed even when inventory and payment work. section 6.8.4

Ask which work must finish while the customer waits. Inventory and payment can remain synchronous, while label printing goes into a queue for later processing. The response no longer waits for that task, but the label is not yet printed when the order is acknowledged. section 6.8.4

Figure 6.14 Avoiding the Distributed Monolith: Breaking a fragile synchronous chain with an asynchronous queue ensures critical tasks succeed even if non-critical services fail.

Compare the fragile synchronous chain with a design that queues non-critical work. section 6.8.4

A quick receipt, a slower analysis

In an AI pipeline simulation, a producer puts a comment event on a channel and immediately reports “Comment Received.” A background consumer waits three seconds to simulate model latency before printing the analysis. The quick receipt and the finished result are different milestones. section 7.7.1

The practical rule: count the waits on the path to the result you care about. A fast acknowledgment can improve the immediate experience, but it is not the same as finishing the work. section 6.8.4 section 7.7.1

How Grokking Software Architecture covers it

How deep it goes
Defines network latency and works through sequential call costs and asynchronous alternatives, without detailing measurement methods.
Examples
Illustrates latency with an order API chain, a delayed AI analysis simulation, and a supply-chain game.
What you can do after
Evaluate synchronous dependencies and identify work that can be queued to shorten the immediate wait.
How it connects
Connects latency with throughput, synchronous communication, queues, availability, and microservices.

Also explained in

Compare how 12 books explain latency →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Software Architecture. The section links point to the parts of the book it draws on.