liveBook · Manning

ExtractField

As explained in Kafka Streams in Action, Second Edition

ExtractField is a Kafka Connect single message transform that extracts one field from a record key or value.

ExtractField is a Kafka Connect single message transform that reduces a record key or value to one selected field. It is useful when an earlier transform creates a Struct but the connector needs the field itself. section 5.4

A typical key transformation chain works as follows:

  • ValueToKey copies the user_name field into the Kafka record key, producing a Struct such as {"user_name":"artv"}. section 5.4
  • ExtractField$Key selects user_name from that Struct, leaving the field value as the record key. section 5.4
  • The transform order matters because each operation changes the data consumed by the next operation. section 5.4
Listing 5.2 The JSON transform configuration
"transforms":"copyFieldToKey, extractKeyFromStruct, maskSsn", #1
 "transforms.copyFieldToKey.type":
       "org.apache.kafka.connect.transforms.ValueToKey", #2
 "transforms.copyFieldToKey.fields":"user_name", #3
 "transforms.extractKeyFromStruct.type":
        "org.apache.kafka.connect.transforms.ExtractField$Key", #4
 "transforms.extractKeyFromStruct.field":"user_name",  #5
 "transforms.maskSsn.type":
        "org.apache.kafka.connect.transforms.MaskField$Value", #6
 "transforms.maskSsn.fields":"ssn",    #7
  "transforms.maskSsn.replacement":"xxx-xx-xxxx" #8

Use the complete JSON configuration to see ExtractField$Key placed between ValueToKey and MaskField. section 5.4

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Shows JSON configuration and transform ordering, but does not descend into ExtractField implementation internals.
Examples
Uses user_name, a Struct containing user_name, and an ExtractField$Key configuration entry.
What you can do after
Operate/configure a connector that extracts a field after constructing a key with ValueToKey.
How it connects
Connects ExtractField with Kafka Connect, SMTs, ValueToKey, MaskField, converters, Structs, and record keys.

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.