Metadata records the current state and configuration of a Kafka cluster, allowing brokers and Kafka Streams instances to coordinate activity. It supports leadership, topic replication, membership, and access decisions. section 2.8
Kafka uses metadata to track:
- Cluster membership, including brokers joining the cluster and unavailable brokers being removed. section 2.8
- Topic configuration, including leaders, partition counts, and topic-specific overrides. section 2.8
- Access control, including which users or software may read from or write to particular topics. section 2.8
- Kafka Streams ownership and application-server locations, enabling queries to reach the instance responsible for a key. section 13.2
Forwarding an interactive query
A Kafka Streams instance receives a query for a key. It hashes the key modulo the partition count, checks metadata to identify the owning instance, and forwards the request using that instance's host and port. section 13.2
Figure 13.5 Determining the correct host for a given query
Follow the query-routing flow from a key to the Kafka Streams instance that owns its partition. section 13.2
