9 Managing Kafka within the Enterprise
This chapter covers
- How Kafka handles configuration, leadership assignments, and state coordination
- Exploring Kafka deployment strategies: on-premise, cloud-based, and hybrid solutions
- Best practices for authentication, authorization, encryption, and protecting dana
Let’s take a look at another key actor in the Kafka ecosystem: the controller quorum. Here, you’ll learn how Kafka manages metadata and coordination so you can size and place controllers, anticipate behavior during incidents, and plan migrations from older architectures (e.g., ZooKeeper) to KRaft. We then compare deployment models—on-prem, cloud, and hybrid—to match latency, cost, and operability. Finally, we make security actionable end to end: authentication (mTLS/SASL), authorization (ACLs), encryption in transit (TLS), data-at-rest protection, and even optional end-to-end encryption.
9.1 Field notes: From prototype to deployment
The team gathered once again in the meeting room, ready for another round of discussion. Max arrived with a sense of urgency.
Max: Alright, team, how are things going? Don’t you think it’s time we stop playing around and finally move this prototype into some sort of environment? You know, make it visible, accessible, so other teams can start working with it?
Rob: You’re absolutely right, Max. And that’s exactly what we want to discuss today. But before we can do that, we need to talk about something important: the budget.