liveBook · Manning

Source point

As explained in Kafka Streams in Action, Second Edition

A source node is the root node of a Kafka Streams topology that consumes records and forwards them into the application.

A source point is the root of a Kafka Streams topology. It consumes event records from Kafka topics, turns their byte-array keys and values into application objects, and forwards those records to downstream processing nodes. A topology can contain processing nodes and sink nodes after its source; a minimal application has a source that reads a topic and a sink that writes results to Kafka. section 6.1 section 6.7

Create a DSL source with `StreamsBuilder.stream(...)`, supplying a topic and, when needed, `Consumed` settings. `Consumed.with(...)` selects key and value Serdes, whose deserializers let the stream process typed records instead of broker bytes. `Consumed` can also configure a timestamp extractor, offset reset, and operator name. A source may subscribe to one topic, a collection of topics, or a topic-name pattern. section 6.3.1

Listing 6.1 Defining the source for the stream
KStream<String, String> simpleFirstStream = builder.stream("src-topic",
Consumed.with(Serdes.String(), Serdes.String()));

This minimal DSL call creates a `KStream<String, String>` source for `src-topic` and assigns String Serdes to both record fields. section 6.3.1

Read, transform, and write a stream

A Yelling app starts at `src-topic`, maps each value to uppercase, and sends the resulting records to `out-topic`. The source is the entry point; `mapValues` is its child processing step, and `to` creates the sink side of the pipeline. section 6.3 section 6.3.1

The Processor API exposes the source more directly with `Topology.addSource(...)`. Unlike the DSL, it requires a node name so Kafka Streams can connect child nodes to their parent. It takes a topic name or pattern and can take a timestamp extractor plus separate key and value deserializers; only the processor name and topic selection are required. section 10.1.1

Mask retail purchases at the source

A retail stream reads `RetailPurchase` records from the `transactions` topic using a String key Serde and custom purchase Serde, then immediately applies `mapValues` to mask credit-card numbers. If a transformation needs the key while retaining it, use `ValueMapperWithKey`; use `map` when producing a new key and value. section 6.4.1

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Works through DSL and Processor API creation, Serdes, topic selection, timestamps, and immediate downstream transformations.
Examples
Uses a text-uppercase Yelling app, masked retail purchases, and a brewery order-routing pipeline.
What it assumes
Builds on Kafka producer and consumer behavior while briefly explaining broker byte records and deserialization.
What you can do after
Build it from scratch with `StreamsBuilder.stream` or `Topology.addSource`, selecting topics and record deserialization.
How it connects
Connects topologies with Kafka topics, KStream, Serdes, TimestampExtractor, processing nodes, and sink nodes.

Also explained in

Compare how 3 books explain source point →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.