Operational complexity is the difficulty of provisioning, configuring, securing, monitoring, maintaining, and recovering Kafka in production. Deployment choices determine how much of this work remains with the team. section 9.2 section 9.2.1
Kafka operational complexity includes:
- Provisioning servers and configuring brokers. section 9.2.1
- Securing authentication with mechanisms such as SASL or SSL. section 9.2.1
- Monitoring performance with observability tools and maintaining disaster recovery through replication across data centers. section 9.2.1
- Balancing infrastructure control, fault tolerance, compliance, cost, and engineering capacity. section 9.2 section 9.2.1
On-premises versus managed Kafka
An on-premises deployment gives a large enterprise with dedicated Kafka engineers control to tune performance and optimize costs, but the team owns setup, security, monitoring, patching, and recovery. A managed service shifts provisioning, updates, hardware, and cluster monitoring to the provider, while limiting broker-level tuning and version control and often increasing cost at scale. section 9.2.1
