This chapter covers
- Evaluating higher-level abstractions versus more control
- Working with sources, processors, and sinks to create a topology
- Digging deeper into the Processor API with a stock analysis processor
- Creating a co-grouping processor
- Integrating the Processor API and the Kafka Streams API
Up to this point in the book, we’ve been working with the high-level Kafka Streams API. It’s a DSL that allows developers to create robust applications with minimal code. The ability to quickly put together processing topologies is an important feature of the Kafka Streams DSL. It allows you to iterate quickly to flesh out ideas for working on your data without getting bogged down in the intricate setup details that some other frameworks may need.
But at some point, even when working with the best of tools, you’ll come up against one of those one-off situations: a problem that requires you to deviate from the traditional path. Whatever the particular case may be, you need a way to dig down and write some code that just isn’t possible with a higher-level abstraction.