Partition reassignment changes which brokers store a topic’s partitions and replicas. It redistributes load across brokers and can change a topic’s replication factor when reliability or capacity requirements change. section 3.1.2 section 3.3
Use the available Kafka tools according to the scope and constraints of the change:
- Use kafka-topics.sh for a reassignment affecting a single topic or for straightforward replica assignment changes. section 3.1.2
- Use kafka-reassign-partitions.sh for complex assignments, multiple topics, or automatic redistribution across topics. section 3.1.2 section 3.3
- Keep the replication factor no higher than the number of brokers, and remember that Kafka cannot reduce a topic’s partition count without risking data loss. section 3.1.2
Redistributing a changelog topic
A products.prices.changelog topic can place partition 0 on brokers 3 and 2, with broker 3 as leader, and partition 1 on brokers 1 and 3, with broker 1 as leader. After increasing the topic to three partitions, partition 2 is assigned to brokers 2 and 3, with broker 2 as leader. The resulting assignments spread leadership and replicas across the broker set. section 3.1.2
$ kafka-topics.sh \
--describe \
--topic products.prices.changelog \
--bootstrap-server localhost:9092
Topic: products.prices.changelog TopicId: 2VvA24HxRwqhF-znSY1tAQ
PartitionCount: 2 ReplicationFactor: 2 Configs:
Topic: products.prices.changelog Partition: 0 Leader: 3
Replicas: 3,2 Isr: 3,2
Topic: products.prices.changelog Partition: 1 Leader: 1
Replicas: 1,3 Isr: 1,3
Inspect the topic description to see partition leaders, replica assignments, and in-sync replicas before planning a reassignment. section 3.1.2
