liveBook · Manning

setConcurrency

As explained in Kafka Streams in Action, Second Edition

setConcurrency is a listener-container-factory method that sets the number of concurrent listener containers or consumer threads.

setConcurrency configures a ConcurrentKafkaListenerContainerFactory with the desired number of concurrent listener containers or consumer threads. Setting it to the input topic’s partition count can provide one KafkaConsumer per partition and maximize throughput. section 12.2.2

Listing 12.11 Increasing the concurrency level for a container listener factory
@Bean
public ConcurrentKafkaListenerContainerFactory<String, LoanApplication>
                    kafkaListenerContainerFactory() {        #1
    ConcurrentKafkaListenerContainerFactory<String, LoanApplication>
           kafkaListenerContainerFactory =
            new ConcurrentKafkaListenerContainerFactory<>();
    kafkaListenerContainerFactory.setConsumerFactory(consumerFactory());
    kafkaListenerContainerFactory.setConcurrency(partitions);    #2
    return kafkaListenerContainerFactory;
}

Configure the consumer factory, set concurrency from the configured partition count, and return the customized listener-container factory. section 12.2.2

Apply the configuration in a Spring Kafka application by:

  • Defining a ConcurrentKafkaListenerContainerFactory bean and calling setConcurrency(partitions). section 12.2.2
  • Naming the bean kafkaListenerContainerFactory to replace Spring Boot’s default without changing listener code. section 12.2.2
  • Using a custom bean name with the @KafkaListener containerFactory attribute when explicit selection is preferred. section 12.2.2

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Demonstrates factory configuration and bean-selection alternatives, but does not detail rebalancing or thread-management internals.
Examples
Uses a loan-application listener factory configured with setConcurrency(partitions), plus default and custom factory names.
What it assumes
Assumes familiarity with Kafka topics, partitions, consumers, Spring beans, and @KafkaListener configuration.
What you can do after
Operate/configure a Spring Kafka listener factory for partition-aligned consumer concurrency.
How it connects
Connects ConcurrentKafkaListenerContainerFactory, KafkaConsumer, Kafka partitions, Spring Boot, and @KafkaListener.
Cover of Kafka Streams in Action, Second Edition
liveBookmanning.com
Read Kafka Streams in Action, Second Edition on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.