liveBook · Manning

Transform

As explained in Kafka Streams in Action, Second Edition

A Kafka Connect transform is a record-level operation that changes records as they pass through a connector.

A Kafka Connect transform changes records as they pass through a connector. It addresses practical stream-ingestion problems such as missing keys and sensitive fields that should be altered or masked before records enter Kafka. section 5.3

Transforms are useful when connector output needs adjustment before publication:

  • Add keys so updates for the same student can remain ordered within a partition instead of being distributed unpredictably. section 5.3
  • Alter or mask students’ social security numbers to limit exposure before the data reaches Kafka. section 5.3

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Identifies missing keys and masking as applications but stops short of implementation details for individual transforms.
Examples
Uses student updates without keys and student social security numbers requiring masking.
What it assumes
Assumes a running connector and basic awareness that Kafka guarantees ordering only within individual partitions.
What you can do after
Apply Kafka Connect transforms to add keys or mask sensitive fields before records reach Kafka.
How it connects
Links transforms with Kafka Connect, connectors, record keys, partitions, ordering, and data masking.
Cover of Kafka Streams in Action, Second Edition
liveBookmanning.com
Read Kafka Streams in Action, Second Edition on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.