Chapter 13. Streaming
In this chapter
- Processing streams of events in bounded memory
- Streaming events over HTTP with akka-http
- Broadcasting and merging with the graph DSL
- Mediating between streaming producers and consumers
In chapter 12 you learned how to integrate Akka applications with external services using requests and responses. In this chapter we’ll look at integrating external services using streams of data.
A stream of data is a sequence of elements that could have no end. Conceptually, a stream is transient in that it only exists as long as there’s a producer providing elements to the stream and a consumer reading elements from the stream.
One of the challenges for applications that consume streams is that you can’t know beforehand how much data you’ll need to handle, because more data may be produced at any time. Another challenge in streaming applications is dealing with the varying speeds of producers and consumers. If your application mediates between streaming producers and consumers, there’s one problem you’ll have to solve: how to buffer data without running out of memory. How can a producer know if a consumer can or can’t keep up?