A processing node is the execution point in a Kafka Streams topology where events are filtered, mapped, joined, aggregated, or transformed. It sits between source nodes that read Kafka topics and sink nodes that publish processed events. section 8.3.3
A topology can be assembled automatically through Kafka Streams DSL functions or defined explicitly with custom processors. Stateful processing nodes connect to state stores that retain intermediate results for joins and aggregations; edges carry data in memory or through Kafka topics. section 8.3.3
Custom processing nodes are useful when standard streaming operators do not express the required behavior:
- Implement custom logic in Java through classes that implement the Processor interface. section 8.3.3
- Access state stores, schedule periodic actions, and interact directly with the Kafka client API. section 8.3.3
- Package reusable processors in a shared library and connect them programmatically within an explicitly defined topology. section 8.3.3
Fraud detection
A custom processing node can inspect incoming transactions, compare them with historical patterns held in state stores, and identify anomalies in real time. section 8.3.3
Figure 8.9 Internally, the Kafka Streams application is represented as a graph. Events flow from input topics through source nodes, are processed by processing nodes, and are finally published to output topics via sink nodes.
Trace the topology visually from input topics through source nodes and processing nodes to output topics through sink nodes. section 8.3.3
