Resource contention occurs when concurrent operations compete for a limited shared resource and some must wait. The resource might be CPU time, memory or I/O capacity, or a lock protecting shared data. The resulting waits increase latency and can be especially visible in the tail of a latency distribution. section 2.3.3 section 5.1
Where contention appears:
- On a virtualized host, several VMs may seek CPU time at once. A neighboring workload can therefore introduce latency variance even when the application itself has not changed. section 2.3.3
- At a lock, concurrent clients must enter a critical section serially. Waiting threads lose the opportunity to make progress in parallel; lock operations and wakeups add further overhead. section 5.1 section 8.2.1
- Across a dataset, partitioning can reduce the number of clients coordinating over one lock. A skewed partitioning strategy can instead create hot partitions and worsen latency. section 5.1
Concurrency helps a server do useful work while one request waits for I/O, but raising concurrency does not remove resource limits. Little’s law gives average latency L = concurrency C / throughput T: if C rises while T stays constant, L rises. For an API server receiving 1,000 requests per second with 20 ms request processing time, the corresponding concurrency is about 50 requests. Use that relationship to reason about load, then measure latency under the intended workload. section 9.1 section 5.1
A contended counter
A counter protected by a mutex has little lock overhead with one thread. As more threads update the same counter, they increasingly wait to enter its critical section. Benchmarking the lock with 1, 10, and 100 threads makes the change visible rather than assuming that adding threads improves performance. section 8.2.1
locking-bench/Mutex::lock()+unlock() (10 threads)
time: [588.11 ns 602.81 ns 617.44 ns]
locking-bench/Mutex::lock()+unlock() (100 threads)
time: [5.1196 µs 5.2806 µs 5.4361 µs]
The mutex lock-and-unlock benchmark rises from roughly 600 ns at 10 threads to 5.3 µs at 100 threads, showing the cost of increased contention in this test. section 8.2.1
