liveBook · Manning

To

As explained in Kafka Streams in Action, Second Edition

KStream.to is a Kafka Streams operation that writes stream records to a Kafka topic through a sink node.

KStream.to writes records from a KStream to a Kafka topic through a sink node. It completes a stream-processing path rather than providing another KStream for chaining, so it is a terminal operation alongside print and foreach. section 6.4.2 section 6.5

A typical sink operation combines these pieces:

  • Transform purchase records into a KStream, such as purchase-pattern records keyed by ZIP code. section 6.4.2
  • Call to with the destination topic name, such as patterns. section 6.4.2
  • Pass Produced.with using the key and value Serde required to serialize output records. section 6.4.2
  • Use print or peek separately when you need development-time visibility before or after the sink path. section 6.5

Writing purchase patterns

A retailer can flatten each transaction into purchased items, map those items into Pattern values, and write the resulting stream to the patterns topic: patternKStream.to("patterns", Produced.with(stringSerde, purchasePatternSerde));. Printing the stream before this call provides console feedback, but to ends the chain. section 6.4.2 section 6.5

For per-record routing, pass a TopicNameExtractor instead of a fixed topic. Its extract method can inspect the key, value, or RecordContext, including timestamps, offsets, partitions, headers, and the source topic. Extracted destination topics must already exist; Kafka Streams does not create them by default. section 6.6.4 section 6.7

Listing 6.11 Patterns processor and sink node that writes to Kafka
KStream<String, Pattern> patternKStream = retailPurchaseKStream
                .flatMap(retailTransactionToPurchases)  #1
                .mapValues(patternObjectMapper);   #2

patternKStream.print(Printed.<String,Pattern>toSysOut()
              .withLabel("patterns-stream"));  #3
patternKStream.to("patterns",
        Produced.with(stringSerde,purchasePatternSerde));  #4

Follow the complete pattern-stream construction from flattening and value mapping through console inspection and the Kafka sink. section 6.4.2

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Explains fixed and dynamically selected destinations, serialization, terminal behavior, and topic prerequisites without detailing broker internals.
Examples
Uses retailer purchase patterns, ZIP-code keys, a patterns topic, department routing, and header-based destination selection.
What it assumes
Assumes a KStream topology, Kafka topics, key and value Serde objects, and basic record metadata such as headers.
What you can do after
Build it from scratch by serializing a processed KStream to a fixed topic or routing each record with TopicNameExtractor.
How it connects
Connects KStream, sink nodes, Kafka topics, Serde, TopicNameExtractor, RecordContext, headers, print, and peek.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.