liveBook · Manning

Metadata

As explained in Apache Kafka in Action

Metadata is descriptive information about Kafka topics, brokers, partitions, replicas, assignments, and leadership.

Kafka metadata describes topics, active brokers, partition assignments, and partition leaders. The controller manages partition and replica state; when a broker fails, it can reassign partitions and select new leaders. section 4.3.1

Clients need metadata to find which broker leads each topic partition. Brokers cache this information, and clients request it from a broker when connecting. A bootstrap server can be any broker, but specifying several brokers—or using load balancing through a load balancer or DNS—avoids relying on one broker to start. section 7.4

Figure 4.11 Kafka uses either a KRaft-based or a ZooKeeper-based coordination cluster. Both should consist of an odd number of nodes (usually three or five) and use a protocol to find a consensus.
A group of ovals with text

AI-generated content may be incorrect.

The diagram contrasts ZooKeeper-based and KRaft-based coordination clusters and notes their consensus setup. section 4.3.1

$ kafka-topics.sh \
    --list \
    --bootstrap-server \
      localhost:9092,localhost:9093,localhost:9094

This topic-listing command shows how to provide multiple bootstrap brokers instead of depending on a single broker. section 7.4

How Apache Kafka in Action covers it

How deep it goes
Explains stored topic and partition information, leader assignment, broker caching, and client metadata requests.
Examples
Uses topic-listing commands with one broker or several bootstrap servers to illustrate metadata discovery.
What you can do after
Operate Kafka clients by supplying bootstrap servers and choosing multiple brokers or load balancing for startup resilience.
How it connects
Connects metadata to Kafka brokers, controllers, partitions, leaders, ZooKeeper, KRaft, and bootstrap-server requests.

Also explained in

Compare how 21 books explain metadata →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.