7 Packaging and Deployment
This chapter covers
- Preparing Kafka Streams applications for production
- Using containers and Kubernetes with Kafka Streams
- Monitoring your applications in the production environment
- Improving the robustness of Kafka Connect
Most existing books cover the core concepts of Apache Kafka but skip the important topic of running it and related applications in production. We think that this topic is of utter importance because there is no justification for any piece of software if it does not provide value to the business. Consequently, the rest of the book focuses on running streaming data pipelines in the production environment. This chapter covers the packaging and deployment of Kafka Streams applications and discusses some of the challenges you will face when running Kafka Connect in production.
Figure 7.1 Common environments (or stages) for developing and operating streaming data pipelines.
As visualized in Figure 7.1, most software architectures for streaming data pipelines consist of three to four tiers