Avro is a binary data format for structured messages. In Kafka, where messages are stored as byte arrays, Avro provides a schema that defines how each payload should be interpreted. section 3.2.2 section 13.1.4
Avro is useful when topics retain data for a long time and multiple schema versions coexist. Its schema makes messages self-describing, supports traceability, and gives producers and consumers room to evolve independently. section 13.1.4 section 13.4
A product schema
A product record can define string fields for id and name, numeric fields for price and stock, and an optional origin field with a null default. This explicit structure keeps records consistent while allowing the optional field to be absent. section 13.1.4
{
"type": "record",
"name": "Product",
"namespace": "com.example.supermarket",
"fields": [
{"name": "id", "type": "string"},
{"name": "name", "type": "string"},
{"name": "price", "type": "float"},
{"name": "stock", "type": "int"},
{"name": "origin", "type": ["null", "string"], "default": null}
]
}
Inspect the Avro record definition to see how field names, types, namespaces, and optional values are declared. section 13.1.4
Sending the schema with every message makes the message much larger: one example uses a 35-byte binary payload and a 292-byte JSON schema. A schema registry avoids that cost by storing schemas centrally and placing only a schema ID in each Kafka message. section 13.1.4
Practical guidance:
- Choose a consistent format across topics rather than selecting a different format for each topic. section 3.2.2
- Document schemas and enforce compatibility checks before accepting changes that could break consumers. section 3.2.2 section 13.4
