KRaft

As explained in Kafka for Architects

KRaft is Kafka's metadata management mode that uses an internal controller quorum instead of ZooKeeper.

A Kafka prototype can run brokers that serve messages, but a production cluster also needs a reliable way to track topics, partitions, broker health, and leadership. KRaft (Kafka Raft) is Kafka’s metadata-management mode: a controller quorum maintains cluster state internally rather than relying on a separate ZooKeeper cluster. Brokers handle client traffic; controllers coordinate the cluster. Depending on the configuration, a server may perform either or both roles. section 1.3.2 section 9.0 section 9.1.4

The controller quorum makes metadata changes and detects failures:

• The active controller records changes such as topic creation, partition assignments, and broker registrations in a metadata log. Follower controllers replicate the log; an update is committed after a majority acknowledges it. section 1.3.2 section 9.1.1
• Brokers send heartbeats to the active controller. If one stops responding within the configured timeout, the controller can treat it as failed. If the active controller fails, a healthy controller can be elected to replace it. section 1.3.2
• Clients obtain metadata through brokers, not by reading the controller log. Brokers can keep a local, read-only metadata copy for faster access and startup. section 9.1.1

Figure 9.2 Metadata updates are sent to the leader controller, which appends each update to its log. Follower controllers fetch these updates from the leader, and once a majority have replicated the update, it is marked as committed.

Follow one metadata update from the leader controller to its followers; majority replication is the point at which the update becomes committed. section 9.1.1

Planning for a data-center outage

A controller placement that loses its majority when one data center goes down cannot continue leader elections or metadata changes. Distributing controllers across three data centers can preserve quorum through a complete outage of one. More generally, tolerating N controller failures requires 2N + 1 controllers, leaving a majority of N + 1. section 9.1.1 section 9.5

KRaft changes operational planning, not just where metadata is stored. Size and place controllers for the failures the cluster must survive, and account for controller behavior in monitoring and incident plans. For an older ZooKeeper-based cluster, migration is an architectural upgrade: Kafka 4.0 requires ZooKeeper’s removal, and the surrounding Kafka components must be checked for compatibility before upgrading. section 9.0 section 9.5 section 11.1

How Kafka for Architects covers it

How deep it goes

Explains controller roles, metadata-log replication, majority commits, and failure handling, but defers step-by-step configuration.

Examples

Illustrates majority commit with a metadata-update diagram and quorum loss with a data-center outage scenario.

What it assumes

Assumes familiarity with Kafka brokers and clusters while defining controller roles and the Raft–KRaft distinction.

What you can do after

Evaluate/understand controller placement, quorum-loss consequences, and the migration implications of removing ZooKeeper.

How it connects

Connects KRaft to Raft consensus, controller quorums, broker heartbeats, metadata logs, and ZooKeeper migration.

liveBookmanning.com

Read Kafka for Architects on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Apache Kafka in ActionKRaft is Kafka's protocol for managing cluster metadata and internal coordination without an external coordination system.

Compare how 2 books explain KRaft →

Related concepts

• Raftimplements · Raft is a distributed consensus algorithm that uses leader election and replicated logs to coordinate nodes and maintain consistent replicated state despite failures.
• ZooKeeperalternative to · Apache ZooKeeper is a distributed coordination service that provides shared configuration, naming, synchronization, metadata management, and leader-election primitives.
• controllerimplemented by · A Kafka controller is a server role that manages cluster metadata and coordinates partition, replica, and leader state.
• coordination clusteralternative to · A coordination cluster is a group of nodes that monitors a distributed cluster and manages its membership, metadata, assignments, and coordination state.
• Apache Kafkaconfigures · Apache Kafka is a distributed event-streaming platform for publishing, storing, processing, and consuming streams of records.

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.