Throughput is the actual rate of successful work or message delivery over time. It measures how much data an application transfers or how many requests it processes per time unit. section 1.4
Throughput differs from bandwidth and latency. Bandwidth sets the communication channel’s maximum capacity; latency measures the time for one request to complete. Throughput records the rate the system actually sustains. section 1.4
Laundry as a pipeline
Washing takes 30 minutes and drying takes 60. Running both steps serially gives 90-minute latency and throughput of 1/90 loads per minute. Pipelining overlaps the next wash with drying, increasing throughput to 1/60 loads per minute while making per-load latency 120 minutes and leaving drying as the bottleneck. section 1.5
Figure 1.2 Processing without pipelining. We first perform step W (washing) fully and only then perform step D (drying). As the time to complete W is 30 minutes and the time to complete D is 60 minutes, washing and drying take 90 minutes in total. Therefore, we say that the latency to wash and dry clothes is 90 minutes and the throughput is 1/90 loads of laundry washed and dried per minute.
Compare serial and pipelined processing to see how overlap changes latency, throughput, and bottlenecks. section 1.5
Use throughput reasoning to:
- Measure actual application data flow or request processing rather than relying only on link capacity. section 1.4
- Increase throughput by adding concurrency, while recognizing that parallelism can trade higher throughput for worse latency. section 1.5
- Analyze throughput together with latency using Little’s law and Amdahl’s law. section 2.0
