liveBook · Manning

Schema reference

As explained in Kafka Streams in Action, Second Edition

A schema reference is a schema-defined link to another schema that enables schemas to share common definitions.

A schema reference links a schema to another schema so multiple definitions can share one reusable type. Reuse removes duplicated nested records and keeps maintenance localized when the shared definition changes. section 3.5

Use schema references to organize related schemas:

  • Move a shared Person record into its own person.avsc file, then reference it from Company and College arrays. section 3.5
  • Keep the referenced schema in the same schema language: Avro references Avro, JSON Schema references JSON Schema, and Protocol Buffers imports Protocol Buffers. section 3.5
  • Register the shared subject first, then attach its subject name and version when registering each referring schema. section 3.5

Constrain several event types on one topic

An online retailer scans packages as TruckEvent, PlaneEvent, or DeliveryEvent. An Avro union lists only those three referenced types, while TopicNameStrategy registers the union under one topic subject. The topic can therefore contain multiple related event types while rejecting types outside the union, preserving their shared sequence. section 3.6

Listing 3.37 Registering the all_events schema with references
subject('truck_event',
      'src/main/avro/truck_event.avsc', 'AVRO')      #1
subject('plane_event','src/main/avro/plane_event.avsc', 'AVRO')
subject('delivery_event','src/main/avro/delivery_event.avsc', 'AVRO')

subject('inventory-events-value',
          'src/main/avro/all_events.avsc','AVRO')     #2
  .addReference("bbejeck.chapter_3.avro.TruckEvent",
                                 "truck_event", 1)          #3
  .addReference("bbejeck.chapter_3.avro.PlaneEvent", "plane_event", 1)
  .addReference("bbejeck.chapter_3.avro.DeliveryEvent", "delivery_event", 1)

Inspect the registration configuration for the shared event schemas and the all_events schema, including each reference’s subject and version. section 3.6

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Demonstrates nested-record replacement, registration configuration, and union-based topic control without detailing registry internals.
Examples
Uses shared Person records for companies and colleges, plus truck, plane, and delivery events in one Avro topic.
What you can do after
Operate/configure schema registration with shared references and restrict a topic to explicitly permitted event types.
How it connects
Connects schema references with Avro, JSON Schema, Protocol Buffers, schema compatibility, unions, and TopicNameStrategy.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.