liveBook · Manning

Consumer assignment strategy

As explained in Kafka for Architects

A partition assignment is the allocation of Kafka topic partitions to consumer instances for reading.

A consumer assignment strategy determines how Kafka distributes topic partitions among consumers in a group. It supports parallel processing while defining which consumer reads each partition. section 4.2.3 section 4.7

During group startup, consumers join a broker-selected coordinator. One consumer becomes group leader, calculates assignments using the configured strategy, and returns them to the coordinator, which informs the group members. Consumers then fetch data directly from partition leaders. section 4.2.3

Design and governance considerations include:

  • Configure the client-side partition.assignment.strategy property to select an available strategy. section 4.2.3
  • Implement the PartitionAssignor interface when a custom distribution strategy is required. section 4.2.3
  • Document the assignment strategy alongside consumer services, expected groups, partition counts, and consumption models. section 10.1.3
  • Distinguish dynamic subscription-based assignments from explicit partition assignment when targeted processing is needed. section 4.7

Consumer-group startup

Several consumers join a group, one is elected leader, and the leader calculates a partition distribution. The coordinator distributes the result; each consumer polls only its assigned partitions, processes messages, and commits offsets. section 4.2.3 section 4.2

Figure 4.9 Distributing the assignments
A screenshot of a black background

Description automatically generated

The assignment-distribution diagram makes the coordinator, group leader, consumers, and partition allocation relationships concrete. section 4.2.3

How Kafka for Architects covers it

How deep it goes
Explains coordinator and leader roles, configuration, custom assignors, and assignment flow without detailing individual strategy algorithms.
Examples
Uses consumer-group startup, leader-based partition distribution, polling, message processing, and offset commits as concrete workflow examples.
What it assumes
Assumes consumer-group and partition concepts while teaching coordinator, leader, joining, assignment, polling, and committing workflows.
What you can do after
Operate/configure consumer groups through partition.assignment.strategy and document assignment choices with topic requirements.
How it connects
Connects consumer groups, group coordinators, group leaders, partition leaders, rebalances, subscriptions, and offset commits.

Also explained in

Compare how 2 books explain consumer assignment strategy →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.