liveBook · Manning

ConcurrentKafkaListenerContainerFactory

As explained in Kafka Streams in Action, Second Edition

ConcurrentKafkaListenerContainerFactory is a Spring Kafka factory for creating listener containers with configurable consumer concurrency and behavior.

ConcurrentKafkaListenerContainerFactory creates the listener containers behind methods annotated with @KafkaListener. A listener container wraps a KafkaConsumer, subscribes it to the declared topic or topics, and delivers records to a message-handling POJO method. Spring Boot can create the factory automatically when Kafka support is enabled, while an application can supply its own factory when default behavior is insufficient. section 12.2.1 section 12.2.2

Its key throughput control is concurrency. With one listener consumer handling an input topic's partitions, increasing partitions alone does not create a consumer per partition. Setting factory concurrency to the configured partition count creates a KafkaConsumer per partition, supporting maximum throughput for that topic. section 12.2.2

Configure a factory for partition-level consumption

Define a ConcurrentKafkaListenerContainerFactory bean, attach the application's ConsumerFactory, and call setConcurrency(partitions). Naming the bean method kafkaListenerContainerFactory replaces the default factory expected by the container. Alternatively, name a factory explicitly and select it from @KafkaListener with containerFactory = "custom-container". section 12.2.2 section 12.2.2

This factory is part of a larger listener flow: @KafkaListener supplies the topic and consumer group, the container consumes records, the handler processes a LoanApplication, and KafkaTemplate can send the processed record to an approved, rejected, or quality-assurance topic. Property placeholders keep topics and group IDs configurable without recompilation. section 12.2.1 section 12.2.1

Listing 12.11 Increasing the concurrency level for a container listener factory
@Bean
public ConcurrentKafkaListenerContainerFactory<String, LoanApplication>
                    kafkaListenerContainerFactory() {        #1
    ConcurrentKafkaListenerContainerFactory<String, LoanApplication>
           kafkaListenerContainerFactory =
            new ConcurrentKafkaListenerContainerFactory<>();
    kafkaListenerContainerFactory.setConsumerFactory(consumerFactory());
    kafkaListenerContainerFactory.setConcurrency(partitions);    #2
    return kafkaListenerContainerFactory;
}

The configuration shows the complete minimal customization: construct the factory, provide its ConsumerFactory, and align concurrency with the partition count. section 12.2.2

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Explains factory creation and concurrency configuration through a bean example, without detailing container internals.
Examples
Uses a loan-application processor and a factory configured with concurrency equal to the partition count.
What it assumes
Introduces listener, consumer, topic, partition, and consumer-group roles alongside Spring Kafka annotations.
What you can do after
Operate/configure it by replacing or naming a listener-container factory and setting consumer concurrency for partitions.
How it connects
Connects @KafkaListener, KafkaConsumer, ConsumerFactory, KafkaTemplate, topics, partitions, and consumer groups.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.