2 Modeling and Measuring Latency
This chapter covers
- Designing with laws of latency in mind
- Thinking of latency as a distribution
- Discovering common sources of latency
- Understanding how latency compounds
- Measuring latency correctly
Now that we have built an intuition around latency in Chapter 1, it’s time to dive into more detail.
In this chapter, we will begin by exploring two principles of latency: Little's Law and Amdahl's Law. Little's Law sheds light on the correlation between latency, throughput, and concurrency, giving us insights into the constraints of our system design. On the other hand, Amdahl's Law delves into the interplay between latency and parallelism. It outlines the potential theoretical acceleration achievable by reducing latency or enhancing parallel processing. We then delve into typical sources of latency, ranging from physics to operating system dynamics, and their impact on your application's latency. We will also investigate how latency compounds and how you can measure latency. Finally, we'll roll up our sleeves and get to work by implementing a simple tool to measure network latency and visualize these measurements effectively.
2.1 Laws of latency
When building for low latency, you must grasp the fundamental boundaries of what you can achieve. In this section, we'll delve into two principles concerning latency: Little's Law and Amdahl's Law, which help you predict how your design impacts your system performance.