A cascading failure begins when one service becomes slow or unresponsive and dependent services accumulate blocked threads, fill connection pools, time out, and fail in succession. The original component may be healthy enough to respond eventually; the wider outage develops because other services are waiting for it. section 12.3 section 12.3.1
Figure 12.5 A failure or slowdown in one service (e.g., due to high latency, blocked resources, or an overloaded dependency) can propagate through the system. Upstream services that rely on it may begin to stall, exhaust thread pools, or trigger retries—eventually causing multiple services to degrade or fail, even if they were functioning correctly on their own.
Use the propagation diagram to connect a single slowdown with upstream stalls, exhausted thread pools, retries, and multi-service degradation. section 12.3.1
Investigate the failure by looking for these signals:
- Check metrics for rising latency, CPU spikes, and error rates, while remembering that systemic failures can also appear as broadly degraded performance. section 12.3
- Inspect thread dumps for many WAITING or TIMED_WAITING threads sharing remote-call stack traces, HTTP client operations, socket reads, or connection-pool paths. section 12.3.1
- Follow trace IDs and spans across services to locate the slow operation and distinguish the originating dependency from services that merely waited. section 12.4
- Compare client and server durations, and look for repeated requests or duplicate request IDs that reveal retry storms and timeout mismatches. section 12.4
Tracing a slow payment dependency
A checkout request takes 850 ms: validation takes 20 ms, storing the session takes 10 ms, and calling the payment service takes 790 ms. The payment span consumes 780 ms, identifying the downstream operation as the likely source of the delay. A matching thread dump may show many request threads waiting in RestTemplate and CompletableFuture calls. section 12.3.1 section 12.3.1
