During my time as a software developer, I’ve had the good fortune to work with current software on exciting projects. I started out doing a mix of client-side and backend work; but I found I preferred to work solely on the backend, so I made my home there. As time went on, I transitioned to working on distributed systems, beginning with Hadoop (then in its pre-1.0 release). Fast-forward to a new project, and I had an opportunity to use Kafka. My initial impression was how simple Kafka was to work with; it also brought a lot of power and flexibility. I found more and more ways to integrate Kafka into delivering project data. Writing producers and consumers was straightforward, and Kafka improved the quality of our system.
Then I learned about Kafka Streams. I immediately realized, “Why do I need another processing cluster to read from Kafka, just to write back to it?” As I looked through the API, I found everything I needed for stream processing: joins, map values, reduce, and group-by. More important, the approach to adding state was superior to anything I had worked with up to that point.
I’ve always had a passion for explaining concepts to other people in a way that is straightforward and easy to understand. When the opportunity came to write about Kafka Streams, I knew it would be hard work but worth it. I’m hopeful the hard work will pay off in this book by demonstrating that Kafka Streams is a simple but elegant and powerful way to perform stream processing.