liveBook · Manning

MaskField SMT

As explained in Kafka Streams in Action, Second Edition

MaskField is a Kafka Connect single message transform that replaces selected record fields with configured or type-based replacement values.

MaskField is a Kafka Connect single message transform that replaces selected record fields with configured values. In a connector pipeline, it runs alongside transformations such as ValueToKey and ExtractField, with the order controlling the final record. section 5.4

Configure the transform by:

  • Adding its name to the connector's comma-separated transforms list. section 5.4
  • Selecting org.apache.kafka.connect.transforms.MaskField$Value as its implementation. section 5.4
  • Specifying the fields to mask and, when needed, a replacement value. section 5.4

Mask an SSN field

A connector can apply MaskField$Value to the ssn field with the replacement value xxx-xx-xxxx, while earlier transforms copy user_name into the key and extract it from the resulting struct. section 5.4

Listing 5.2 The JSON transform configuration
"transforms":"copyFieldToKey, extractKeyFromStruct, maskSsn", #1
 "transforms.copyFieldToKey.type":
       "org.apache.kafka.connect.transforms.ValueToKey", #2
 "transforms.copyFieldToKey.fields":"user_name", #3
 "transforms.extractKeyFromStruct.type":
        "org.apache.kafka.connect.transforms.ExtractField$Key", #4
 "transforms.extractKeyFromStruct.field":"user_name",  #5
 "transforms.maskSsn.type":
        "org.apache.kafka.connect.transforms.MaskField$Value", #6
 "transforms.maskSsn.fields":"ssn",    #7
  "transforms.maskSsn.replacement":"xxx-xx-xxxx" #8

Inspect the complete JSON transform chain, including transform order, implementation classes, target fields, and the SSN replacement. section 5.4

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Explains JSON configuration, pipeline placement, and ordering but does not detail MaskField implementation internals.
Examples
Uses an SSN field replaced by xxx-xx-xxxx in a chain that also derives a user_name record key.
What it assumes
Assumes basic connector configuration while demonstrating the transform setup without requiring Java code.
What you can do after
Operate/configure it by adding MaskField$Value to an ordered connector transform chain and specifying fields and replacements.
How it connects
Connects MaskField with Kafka Connect, single message transforms, ValueToKey, ExtractField, converters, and connector pipelines.
Cover of Kafka Streams in Action, Second Edition
liveBookmanning.com
Read Kafka Streams in Action, Second Edition on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.