Broker.rack property

As explained in Apache Kafka in Action

broker.rack is a Kafka broker configuration that assigns a rack or location identifier to a broker.

broker.rack assigns each Kafka broker a rack, availability zone, data center, or other location identifier. Kafka uses these identifiers to spread replicas across fault domains and improve tolerance of localized failures. section 16.1.4

Location awareness also supports locality for consumers. Since KIP-392, a consumer configured with client.rack can fetch from a nearby replica instead of always fetching from the partition leader; producers still write to the leader. section 9.1.2 section 16.1.4

Apply broker.rack with these operational considerations:

• Set location identifiers on all brokers before creating topics so Kafka can distribute new topic replicas as evenly as possible across locations. section 16.1.4
• Use distinct, genuinely independent racks or availability zones when possible; spreading brokers across labels that share infrastructure does not provide the same fault tolerance. section 16.1.4
• Expect existing topics to require a separate redistribution tool such as Cruise Control; broker.rack alone does not rebalance their replicas. section 16.1.4
• Balance resilience against network cost: stretched clusters across availability zones increase inter-zone traffic, while client.rack can reduce consumer traffic between locations. section 16.1.4

A stretched cluster across availability zones

Assign brokers location identifiers corresponding to their availability zones. When a topic is created, Kafka attempts to distribute its replicas across those identifiers. Consumers can set client.rack to fetch from a same-zone replica, while producers continue sending records to the partition leader. section 9.1.2 section 16.1.4

Figure 9.1 The consumers are consuming from the nearest replica (in the same rack), but producers still have to produce to the leader of the partition.

Visualizes consumers reading from the nearest replica while producers still communicate with the partition leader. section 9.1.2

Figure 16.2 In a stretched cluster, our Kafka brokers are distributed among multiple data centers.

Visualizes brokers distributed across multiple data centers in a stretched Kafka cluster. section 16.1.4

How Apache Kafka in Action covers it

How deep it goes

Explains location labeling, new-topic placement, consumer locality, and existing-topic limitations, but does not provide configuration commands.

Examples

Uses racks, data centers, availability zones, stretched clusters, and nearest-replica consumption as concrete deployment examples.

What it assumes

Assumes Kafka brokers, topics, partitions, replicas, and leaders, while defining rack identifiers through deployment examples.

What you can do after

Operate/configure broker location identifiers and evaluate their effects on replica resilience, consumer traffic, and operating cost.

How it connects

Connects broker.rack to replicas, fault domains, client.rack, KIP-392, partition leaders, availability zones, and Cruise Control.

liveBookmanning.com

Read Apache Kafka in Action on liveBook Get the book at manning.com

Related concepts

• replicaconfigures · A replica is a copy of data or state, or an equivalent running instance of a component, maintained alongside others in a replicated system.
• fault toleranceconfigures · Fault tolerance is the ability of a system to continue providing correct or acceptable service despite failures in its components or operating environment.
• client.rackconfigured by · client.rack is a Kafka client configuration that assigns a rack or location identifier used to select a nearby replica for fetching records.

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.