liveBook · Manning

Union type

As explained in Kafka Streams in Action, Second Edition

A union type is a construct whose value or field may conform to one of several specified types.

A union type allows a value, field, or schema to conform to one of several specified types. In Kafka event streams, an Avro union can keep related event kinds—such as PlaneEvent, TruckEvent, and DeliveryEvent—on one topic while restricting values to the declared alternatives. section 3.6

A practical union design combines three decisions:

  • Declare every permitted event type in one union schema, preserving related events on a shared topic and their occurrence order. section 3.6
  • Register the individual schemas as references, then register the union schema under the topic’s value subject. section 3.6
  • Configure Avro producers with auto.register.schemas=false and use.latest.version=true so serialization uses the union instead of overwriting it with an individual schema. section 3.6
  • Configure Avro consumers with KafkaAvroDeserializer and SpecificRecord, then inspect the concrete record type before processing it. section 4.6.2

Dispatching transportation events

A consumer can branch on the deserialized value: PlaneEvent goes to plane handling, TruckEvent to truck handling, and DeliveryEvent to delivery handling. An alternative Avro design embeds the union in a TransportationEvent record, but then the union field returns Object and requires instance checks at the field level. section 4.6.2

Listing 3.36 Avro schema all_events.avsc with multiple events
[
  "bbejeck.chapter_3.avro.TruckEvent",      #1
  "bbejeck.chapter_3.avro.PlaneEvent",
  "bbejeck.chapter_3.avro.DeliveryEvent"
]

Inspect the compact Avro schema that lists the permitted event classes in the union. section 3.6

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Explains schema declaration, reference registration, serializer settings, consumer configuration, and concrete-type dispatch.
Examples
Uses PlaneEvent, TruckEvent, DeliveryEvent, an all_events.avsc union, and a TransportationEvent record.
What it assumes
Assumes named event schemas and Kafka producer-consumer configuration, while explaining union declarations and concrete-type checks.
What you can do after
Operate Avro producers and consumers that serialize, retrieve, and dispatch multiple event types from one Kafka topic.
How it connects
Connects Avro unions with schema references, Kafka topics, serialization, deserialization, and Protocol Buffers oneof fields.

Also explained in

Compare how 10 books explain union type →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.