Metadata API

As explained in Kafka for Architects

A metadata request is a Kafka client request that retrieves cluster metadata such as partition leadership and broker assignments.

A metadata request lets a Kafka producer discover which brokers lead each partition. The producer uses this broker-to-partition mapping to send records directly to the correct partition leaders rather than routing blindly through the cluster. section 3.2

Because leadership changes infrequently, the producer caches the mapping. The metadata.max.age.ms setting controls refresh timing and defaults to five minutes. A leadership change can make cached routing stale; the broker returns an error, after which the client refreshes metadata and retries the batch. section 3.2 section 2.3.1

Handling a partition-leader change

Suppose a producer has cached partition leadership and sends a batch to the former leader. That broker reports that leadership has changed. The client library retrieves current metadata, identifies the newly elected leader, and resends the batch; the temporary error is recorded in the application log. section 3.2 section 2.3.1

Figure 3.6 The process of receiving and caching metadata

Shows the request, broker mapping, producer-side cache, and metadata refresh cycle. section 3.2

How Kafka for Architects covers it

How deep it goes

Explains discovery, caching, refresh timing, and recovery from leadership changes without detailing protocol encoding.

Examples

Uses cached broker mappings, five-minute refreshes, stale-leader errors, and batch resends as concrete operational examples.

What you can do after

Diagnose producer routing failures by recognizing stale metadata and the client refresh-and-retry response.

How it connects

Connects producers with partition leaders, brokers, metadata caching, leader election, and replica assignments.

liveBookmanning.com

Read Kafka for Architects on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Apache Kafka in ActionA metadata request is a Kafka client request that retrieves information about topics, partitions, brokers, and partition leaders.

Compare how 2 books explain metadata API →

Related concepts

• clientused by · A client is a person, organization, device, or software component that receives or uses a product, service, resource, or functionality provided by another entity.
• producerused by · A producer is an entity or process that creates or supplies values, data, messages, events, or work for another entity to consume.
• metadataproduces · Metadata is information that describes, identifies, or provides context about other data, resources, or artifacts.
• Kafka brokerproduces · A Kafka broker is a server process in an Apache Kafka cluster that stores records and handles producer, consumer, and other client requests.
• partitionproduces · A partition is a distinct subdivision of a larger data, storage, or work unit that can be stored, managed, or processed separately.

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.