A Kafka Streams task is an independently executable processing unit. It combines assigned input partitions, the state stores used for intermediate or aggregated results, and an instance of the processing graph. section 8.3.4
Tasks provide the unit of distribution and parallelism. Kafka Streams creates tasks from input-topic partitions; with multiple inputs, the topic having the most partitions determines the task count. section 8.3.4
Task architecture has several operational consequences:
- Tasks can run on separate threads within one application instance or across multiple instances. section 8.3.4
- Each task processes events through its graph one at a time, emitting results before consuming the next event. section 8.3.4
- Tasks remain independent, so failure in one task or instance does not directly affect the others. section 8.3.4
- When an instance fails, Kafka Streams reassigns the task’s partition assignment, state stores, and graph topology to a healthy instance. section 8.3.4
Partition-driven task count
For input topics Topic1 through TopicN, taskCount equals the maximum partition count among those topics. Increasing partitions can therefore increase available task parallelism, while the task still carries its assigned state stores and topology. section 8.3.4
