Little’s law connects three averages in a computing system: concurrency C, throughput T, and latency L. Its practical form is C = T × L, or L = C / T. Here, concurrency is the average number of requests or operations in the system, throughput is the rate completed, and latency is the average time each spends there. section 2.1.1 section 9.1
Estimate requests in flight
At 1,000 requests per second and 50 milliseconds (0.05 seconds) of average latency, C = 1,000 × 0.05 = 50 requests. The result describes an average number in the system, not a count of requests actively using a processor. section 2.1.1
At fixed throughput, more concurrency corresponds to higher average latency. Requests may wait when they compete for CPU, memory, or I/O bandwidth. To reason about a latency target, estimate the concurrency it implies, then consider whether throughput can improve or the amount of queued work needs managing. Treat the calculation as a model: measure latency to check its assumptions, and inspect the latency distribution as well as its average. section 2.1.1 section 9.1 section 2.1 section 2.7
Figure 2.1 Latency vs. concurrency when throughput varies. The graph shows latency (y axis) as a function of throughput and concurrency (x axis).
The plot shows how the relationship between concurrency and latency changes at different throughput rates. section 2.1.1
