Raft is a distributed consensus protocol for coordinating replicated state and reaching agreement across cluster nodes. Kafka uses it through KRaft, replacing ZooKeeper-based coordination with an integrated controller cluster. section 7.1 section 7.5
KRaft reduces operational complexity by removing the need to run Kafka and ZooKeeper as separate distributed systems. Controllers coordinate the cluster while brokers handle normal Kafka workloads. section 7.1 section 7.5
In a KRaft cluster:
- Controller nodes form the coordination cluster and elect an active controller. section 7.1
- The active controller monitors brokers, manages partition assignments, and performs leader assignments and failover. section 7.1 section 7.5
- The __cluster_metadata topic stores consistent metadata across brokers. section 7.5
- An odd number of controllers supports failure tolerance: one controller suits local development, three tolerate one failure, and five tolerate two. section 7.1
Choosing a controller count
A local Kafka setup can use one controller-broker, but it has no failover guarantee. A production-oriented setup can use three controller-brokers to survive one controller failure; five can survive two. section 7.1
Figure 7.1 With Kafka 3.3, ZooKeeper was replaced by a coordination cluster based on the Raft consensus protocol.
Use the architecture figure to see how Kafka's coordination cluster based on Raft replaces ZooKeeper. section 7.1
