A sub-topology is a portion of a Kafka Streams topology that shares one source node. Every processor downstream of that source belongs to the same sub-topology. section 7.2.3 section 6.6.3
Kafka Streams commonly creates a new sub-topology during repartitioning. It writes records with their updated keys to a topic through a sink node, then reads them through a new source node. Processing after that source belongs to the new sub-topology. section 7.2.3
Use key dependencies to predict when the topology gains another sub-topology:
- A key-changing operation followed by groupByKey, aggregation, or join can trigger repartitioning. section 7.2.3
- Changing a key without a downstream key-dependent operation can leave the topology unchanged; filtering, peeking, or writing does not require repartitioning by itself. section 7.2.3
- A topology with a second independent source appears as another sub-topology. section 6.6.3
Reading a topology description
A description can show Sub-topology: 0 with a source reading src-topic, a processor receiving from that source, and a sink writing out-topic. The arrows expose record flow and processor lineage. section 6.6.3
Figure 7.6 Adding a sink and source node for repartitioning creates a new sub-topology.
Inspect the repartitioned topology to see how adding a sink and source creates a new sub-topology. section 7.2.3
