A Kafka cluster needs a reliable record of changes to its state. In KRaft mode, controllers keep that record in a replicated metadata log. It records changes such as topic creation, partition assignments, and broker registrations; brokers handle client traffic rather than storing this controller log. section 1.3.2 section 9.1
The log is stored as Raft log files on controller filesystems. It is sometimes called the `__cluster_metadata` topic, but it is not an ordinary topic stored on brokers: controllers participate in its Raft quorum and write to it. Brokers may keep a local, read-only copy for faster metadata access. section 9.1.1
A topic creation request
An admin client sends a request to a broker, which forwards the metadata operation to the leader controller. The leader appends the update; follower controllers fetch it. The update is committed after a majority replicate it, and brokers fetch metadata updates to stay synchronized. Regular clients obtain metadata through brokers, not directly from the log. section 9.1 section 9.1.1
Figure 9.2 Metadata updates are sent to the leader controller, which appends each update to its log. Follower controllers fetch these updates from the leader, and once a majority have replicated the update, it is marked as committed.
Follow an update from the leader’s log through follower replication to majority commit. section 9.1.1
This quorum requirement protects committed metadata across controller leader changes, but it also shapes availability: without a controller majority, leader elections and metadata changes halt. Controller placement therefore belongs in high-availability planning. Periodic snapshots capture committed state, while an in-memory cache supports rapid access. section 9.1.1 section 9.5
