Single message transformation
A single message transformation is a stateless Kafka Connect operation that independently modifies or filters each record passing through a connector.
Kafka Streams in Action, Second Edition
A Single Message Transform (SMT) is a lightweight Kafka Connect transformation that modifies each record independently as it passes through a connector.
- How it is used
- Applies lightweight, record-by-record changes while Kafka Connect imports data into Kafka or exports it to external systems.
- How deep it goes
- Explains connector placement, ordered configuration, and practical limits, but stops short of implementing a custom transformation.
- Examples
- Demonstrates credit-card masking, customer-key extraction, field removal, filtering, and nested-structure flattening.
- What you can do after
- Operate/configure Kafka Connect transforms using ordered JSON without writing code for provided SMTs.
Read how Kafka Streams in Action, Second Edition explains single message transformation →
Apache Kafka in Action
A single message transformation is a Kafka Connect transformation that modifies an individual record as it passes through a source or sink connector.
- How it is used
- Applies focused record changes while moving data between external systems, Kafka topics, and connectors.
- How deep it goes
- Explains execution placement, ordered JSON configuration, and common transformations, but stops short of complex ETL design.
- Examples
- Demonstrates ReplaceField, ValueToKey, ExtractField, field renaming, field removal, and product_id key extraction.
- What you can do after
- Configure ordered SMT pipelines through Kafka Connect’s REST API for simple source or sink record changes.
Read how Apache Kafka in Action explains single message transformation →
Related concepts
- sink connectorused by · A sink connector is a Kafka Connect connector that reads records from Kafka and exports them to an external destination system.
- source connectorused by · A source connector is a Kafka Connect connector that reads data from an external source system and publishes records to Kafka.
- converteruses · A converter is a component or operation that transforms data, values, or objects from one type or representation into another.
- ValueToKeyincludes · ValueToKey is a Kafka Connect single message transformation that constructs a record key from selected fields in the record value.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.