A supermarket publishes product records to a Kafka topic. Each record has an ID, name, price, and stock level. A schema makes that expected structure explicit: it is a blueprint for the data that producers write and consumers read. Even without a documented schema, a consumer still assumes one. If the data violates that assumption, the consumer may fail—or silently calculate a wrong result. section 13.1.1 section 13.1
Schemas matter for binary formats such as Avro and Protobuf, which need one for interpretation. They can also describe JSON or XML. Choose a consistent data format across topics, and keep each topic’s schema discoverable and documented, whether in a wiki, Git repository, or dedicated platform. section 3.2.2 section 13.1.1
A product schema becomes a producer–consumer contract when requirements change:
- Adding a category may be harmless to consumers that ignore unknown fields, but a new consumer reading older records will not find it there. section 13.1.1
- Removing stock can break existing consumers; adding a currency to price can affect both old and new consumers. section 13.1.1
- Decide early which changes are compatible and which require agreement. Optional fields can legitimately appear in some messages and be absent from others. section 13.1.1 section 3.2.2
{
"type": "record",
"name": "Product",
"namespace": "com.example.supermarket",
"fields": [
{"name": "id", "type": "string"},
{"name": "name", "type": "string"},
{"name": "price", "type": "float"},
{"name": "stock", "type": "int"},
{"name": "origin", "type": ["null", "string"], "default": null}
]
}
This Avro product schema names each field and its type. Its origin field permits null and defaults to null, illustrating how a schema can accommodate an optional value. section 13.1.4
Avro can pair a schema with each payload, making a message self-describing. Repeating that schema is costly: one illustrated payload is 35 bytes while its schema is 292 bytes. A schema registry avoids sending the full schema each time; the Kafka message carries a schema ID instead. This keeps messages smaller while identifying the schema used for each message, even when versions coexist in a topic. section 13.1.4
