liveBook · Manning

Consumer assignment strategy

As explained in Apache Kafka in Action

A partition assignment strategy is a consumer setting that selects the algorithm used to assign topic partitions among members of a consumer group.

A consumer assignment strategy determines how a Kafka consumer group divides topic partitions among its members. Set it with the consumer property `partition.assignment.strategy`. During a rebalance, the group leader determines the assignments and sends them to the group coordinator; the other consumers follow that decision. Every consumer in the group should use the same strategy, or assignments may change unpredictably and cause processing or performance problems. section 9.4.2

Choose a strategy based on what must stay together and how much work each consumer should receive:

  • **Range Assignor** is the stated default. It places the same partition number from different topics on the same consumer. This helps when related records across topics need to be processed together, but it can leave consumers idle. section 9.4.2
  • **Round Robin Assignor** spreads partitions more evenly when cross-topic joins are not needed. section 9.5
  • **Cooperative Sticky Assignor** improves rebalance behavior by moving partitions in stages rather than stopping the entire group for one rebalance. Other consumers can continue consuming during the process. section 9.4.2 section 15.3.3

Two topics, three consumers

Suppose `products.prices.changelog` and `products.sales` each have two partitions, and three consumers subscribe to both. With Range Assignor, one consumer gets partition 0 of both topics, another gets partition 1 of both, and the third gets nothing. Matching product keys can therefore bring related records to the same consumer. For an actual join, use a stream-processing library such as Kafka Streams rather than building a naive join in a consumer. section 9.4.2

Figure 9.7 The Cooperative Sticky Assignor avoids stopping the world and rebalances by replacing a single rebalance by two consecutive ones. During the first rebalance, consumers are asked to let go of the partitions that should be moved to another consumer; in the second rebalance, the now dangling partitions are assigned to the consumers.
A screenshot of a computer game

AI-generated content may be incorrect.

The two-stage diagram shows how Cooperative Sticky Assignor first releases partitions that must move, then assigns them in a subsequent rebalance. section 9.4.2

After choosing a strategy, watch assigned-partition counts for idle consumers and rebalance metrics for frequent, failed, or slow rebalances. Depending on the strategy, a group may be unable to consume new messages while rebalancing. section 15.3.3

How Apache Kafka in Action covers it

How deep it goes
Explains leader-led assignment, Range placement, and staged cooperative rebalancing; stops short of a configuration walkthrough.
Examples
Shows two two-partition topics assigned to three consumers, leaving one idle under Range Assignor.
What it assumes
Introduces fetch-based consumption and offsets before discussing consumer-group partition assignment; states no explicit prior-Kafka requirement.
What you can do after
Operate/configure Kafka consumers by choosing a consistent group strategy and monitoring assigned partitions and rebalances.
How it connects
Links consumer groups and rebalances to stream joins, Kafka Streams, and consumer metrics.

Also explained in

Compare how 2 books explain consumer assignment strategy →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.