In this chapter
- an introduction to backpressure
- when backpressure is triggered
- how backpressure works in local and distributed systems
“Never trust a computer you can’t throw out a window.”
—Steve Wozniak
Be prepared for unexpected events is a critical rule when building any distributed systems, and streaming systems are not exceptions. In this chapter, we are going to learn a widely supported failure handling mechanism in streaming systems: backpressure. It is very useful for protecting a streaming system from breaking down under some unusual scenarios.
In chapter 4, the team built a stream processing system to process transactions and detect credit card fraud. It works well, and customers are happy so far. However, the chief has a concern—a very good one.
Before moving forward, let’s review the structure of the system to refresh our memory.
The reason streaming systems are increasingly being used is the need for on-demand data, and on-demand data can be unpredictable sometimes. Components in a streaming system or a dependent external system, such as the score database in the diagram, might not be able to handle the traffic, and they also might have their own issues occasionally. Let’s look at a few potential issues that could arise in the fraud detection system.
