When a new consumer joins a Kafka consumer group, partitions may need to move between consumers. A conventional rebalance can pause consumption across the group, even when most assignments will not change. Cooperative Sticky Assignor minimizes assignment changes and lets consumers keep reading partitions that are unaffected by the move. section 9.4.2
Sticky Assignor tries to retain existing assignments, but still stops consumption during a rebalance. Cooperative Sticky Assignor adds a cooperative protocol: consumers first release partitions that must move, then a subsequent rebalance assigns those released partitions to their new consumers. Consumption pauses for the moving partitions, not the entire group. section 9.4.2
A third consumer joins
Two consumers are reading three partitions. When a third consumer joins, the first rebalance removes Partition 3 from Consumer 2. A second rebalance assigns that partition to Consumer 3. The other partitions continue to be consumed while Partition 3 changes hands. section 9.4.2
Figure 9.7 The Cooperative Sticky Assignor avoids stopping the world and rebalances by replacing a single rebalance by two consecutive ones. During the first rebalance, consumers are asked to let go of the partitions that should be moved to another consumer; in the second rebalance, the now dangling partitions are assigned to the consumers.
Follow the two consecutive rebalances: first release the partition that must move, then assign it to its new consumer. section 9.4.2
Use CooperativeStickyAssignor as the consumer-group assignment strategy when reducing rebalance disruption matters. Still monitor rebalance counts, failures, and latency: frequent or slow rebalances can indicate group problems. Check assigned-partition metrics too, especially if a consumer receives no partitions for an extended period. section 9.4.2 section 15.3.3
