15 Stream processing and incremental I/O

We mentioned in the introduction to part 4 that functional programming is a complete paradigm. Every imaginable program can be expressed functionally, including programs that interact with the external world─but it would be disappointing if the IO type were the only way of constructing such programs. IO and ST work by simply embedding an imperative programming language into the purely functional subset of Scala. While programming within the IO monad, we have to reason about our programs, much like we would in ordinary imperative programming.

We can do better. In this chapter, we’ll show how to recover the high-level compositional style developed in parts 1–3 of this book, even for programs that interact with the outside world. The design space in this area is enormous, and our goal here is not to explore it completely but to convey ideas and give you a sense of what’s possible. We’ll build up a compositional streaming API that’s similar to the Functional Streams for Scala (FS2) library (https://fs2.io).

15.1 Problems with imperative I/O: An example

15.2 Simple stream transformations

15.2.1 Creating pulls

15.2.2 Composing stream transformations

15.2.3 Processing files

15.3 Extensible pulls and streams

15.3.1 Effectful streaming computations

15.3.2 Handling errors

15.3.3 Ensuring resource safety

15.3.4 Dynamic resource allocation

15.4 Applications

15.5 Conclusion

Summary

15.6 Exercise answers