Backpressure regulates asynchronous data flow by causing producers to slow or pause when consumers cannot process incoming work at the same rate. It stabilizes concurrency, protects resources, and keeps performance predictable. section 10.5
Figure 10.4 Backpressure controls the flow of work from producer to consumer to avoid overwhelming the consumer. Clients push work to a server, which buffers it. The service itself pulls work from the buffer. The service also signals the clients about the buffer capacity limits so that the clients know when to slow down to avoid overwhelming the system.
Clients push requests to a service buffer; the service pulls work for processing and signals remaining buffer capacity so clients can adjust their sending rate. section 10.5
Apply backpressure through several complementary controls:
- Throttle producers when the consumer approaches capacity, propagating the signal to the layer best able to reduce work. section 10.5.1
- Use a TCP transmission window or delay socket reads until processing capacity exists, allowing the receiver to regulate senders. section 10.5.1
- Buffer work to absorb temporary load spikes, while sizing buffers against the trade-off between latency and dropped work. section 10.5.2
- Rate-limit clients over a time window, and retain dropping as a last-resort safeguard when other controls fail. section 10.5.3
Rate-limited API service
An API server may permit 100 requests per second per client. Requests beyond the time-window limit produce errors or are dropped when the service is already at capacity; choosing the limit requires understanding system capacity and client concurrency. section 10.5.3
