liveBook · Manning

Transform chain

As explained in Kafka Streams in Action, Second Edition

A transform chain is an ordered sequence of record transformations in which each transformation receives the preceding transformation's output.

A transform chain is an ordered sequence of Kafka Connect single-message transforms. Each transform receives the preceding transform’s output, allowing several operations on one record. section 5.4

A practical chain can reshape and protect an incoming database record:

  • ValueToKey copies the user_name field into the record key. section 5.4
  • ExtractField$Key extracts user_name from the intermediate key, leaving the raw value as the key. section 5.4
  • MaskField$Value replaces the ssn field with xxx-xx-xxxx, or with an empty type-appropriate value when no replacement is configured. section 5.4

Configured chain

Configure transforms as copyFieldToKey, extractKeyFromStruct, maskSsn. The intermediate structure becomes Struct {"user_name" : "artv"} before extraction, while the value’s ssn field is masked.

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Explains sequential field operations through configuration and intermediate output, while recommending Kafka Streams beyond roughly two transforms.
Examples
Uses user_name key extraction, an intermediate Struct, and SSN masking with a configured replacement.
What it assumes
Assumes familiarity with Kafka Connect records, fields, keys, values, and connector configuration.
What you can do after
Apply Kafka Connect configurations to chain field-copying, key-extraction, and value-masking operations.
How it connects
Connects Kafka Connect with Kafka Streams, JDBC connectors, record keys, structs, and single-message transforms.
Cover of Kafka Streams in Action, Second Edition
liveBookmanning.com
Read Kafka Streams in Action, Second Edition on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.