Schema Registry centralizes event schemas so producers and consumers can share an explicit data contract. Kafka brokers store records as bytes and do not interpret them; Schema Registry supplies the missing format agreement, validation, versioning, and compatibility checks as applications evolve independently. section 1.6 section 3.2.1
A schema defines the structure of a record: its fields, types, and sometimes defaults or required values. Schema Registry is the source of truth that clients discover when serializing and deserializing. It supports Avro, Protocol Buffers (Protobuf), and JSON Schema, along with serializers, deserializers, and Kafka Streams Serdes for those formats. section 3.2 section 3.7
Figure 3.3 Schema Registry ensures consistent data format between producers and consumers.
Follow the record path: a producer serializer obtains and caches a schema, sends serialized bytes to Kafka, and a consumer deserializer obtains and caches the schema needed to reconstruct the object. section 3.2.1
Schema Registry runs outside Kafka brokers as a distributed application with a REST API. It persists schemas in Kafka's compacted, single-partition _schemas topic. A primary node writes registrations and updates; secondary nodes forward writes to the primary while every node can serve reads. Serializers and deserializers cache schemas locally, avoiding a registry fetch when the needed schema is already cached. section 3.2.3 section 3.9
Use Schema Registry as part of the development flow:
- Start a local Kafka and Schema Registry environment with Docker Compose. section 3.2.2
- Define an Avro, Protobuf, or JSON Schema file, then register it under a subject through the REST API or a build plugin. section 3.2.5 section 3.2.6
- Generate domain objects from Avro or Protobuf schemas; use an open-source generator for JSON Schema. Keep schemas in Git so teams regenerate matching objects after changes. section 3.2.6
- Configure Kafka producers and consumers with Schema Registry-aware serializers or deserializers and the registry URL. section 3.2.9
- Choose a subject name strategy to set the scope of compatibility checks, and control serializer auto-registration where needed. section 3.3
A subject is the namespace used to find an existing schema and assess a new version against it. TopicNameStrategy, RecordNameStrategy, and TopicRecordNameStrategy set that scope. Compatibility modes include backward, forward, and full; their transitive variants check against all prior versions rather than only the immediate predecessor. section 3.3 section 3.9
Schema references let schemas reuse previously registered types instead of repeating their definitions. Register the referenced schema first, then register dependent schemas with a reference name, subject, and version. This supports composed domain models and schemas containing multiple event types. section 3.5 section 3.6
Avenger event contract
Define an Avenger with name, real name, and a movies list; register the Avro schema; generate Java classes; build an Avenger object; then produce and consume it using KafkaAvroSerializer and KafkaAvroDeserializer configured with the registry URL. The generated object conforms to the registered record structure. section 3.2.5 section 3.2.9
Schema Registry is optional because Kafka clients can use custom Serializer and Deserializer implementations. That route requires serialization code and loses registry validation. Registry-aware schemas can instead be shared across Kafka producers, consumers, Kafka Streams, Kafka Connect, and ksqlDB. section 3.8 section 6.4.5 section 11.5
In Kafka Streams, create and configure a Schema Registry-aware Serde with the registry URL. In ksqlDB, schemas can supply field names and types when creating streams or tables, although explicit columns remain necessary when selecting only a subset. Persistent queries inherit their source data format unless overridden. section 6.4.5 section 11.5
