liveBook · Manning

Metadata API

As explained in Apache Kafka in Action

A metadata request is a Kafka client request that retrieves information about topics, partitions, brokers, and partition leaders.

A metadata request gives a Kafka client the cluster information needed to connect to the right broker. Before producing or consuming messages, the client must discover topics, partitions, brokers, and the leader responsible for each partition. section 7.4 section 9.1

The client begins with a bootstrap server, sends a metadata request for its topics, and receives broker and partition-leader information. It then connects directly to the relevant leaders to fetch or produce messages. If leadership changes after a broker failure, the client can issue another metadata request and discover the new leaders. section 7.4 section 7.5

Listing topics through a bootstrap server

A simple test setup can query one broker with: $ kafka-topics.sh --list --bootstrap-server localhost:9092. Any broker can provide the initial metadata, but relying on one address alone creates a startup dependency: clients may fail to start if that broker is unavailable. section 7.4

Using multiple bootstrap servers

A more resilient configuration lists several brokers: $ kafka-topics.sh --list --bootstrap-server localhost:9092,localhost:9093,localhost:9094. As clusters grow, a load balancer or DNS can distribute the initial metadata request, while each broker still needs a reachable address for later connections. section 7.4

How Apache Kafka in Action covers it

How deep it goes
Explains request flow, cached cluster information, leader discovery, and failure recovery without detailing the request protocol.
Examples
Demonstrates kafka-topics.sh with one broker, multiple bootstrap servers, and a figure showing client-to-leader discovery.
What it assumes
Assumes basic Kafka cluster, broker, topic, partition, and consumer concepts while showing the connection sequence concretely.
What you can do after
Operate/configure Kafka clients with resilient bootstrap addresses and understand how they recover connections after leader changes.
How it connects
Ties bootstrap servers to Kafka brokers, topics, partitions, partition leaders, consumers, producers, and DNS-based load balancing.

Also explained in

Compare how 2 books explain metadata API →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.