Serialization converts structured event data into binary byte arrays for storage or transmission. Kafka producers serialize records before sending them, while consumers deserialize fetched records; the broker stores and forwards bytes without interpreting their data type or language. section 3.0
A schema makes the serialized object’s structure explicit by naming the object, fields, and field types. It acts as a contract between producers and consumers, while Schema Registry centralizes schemas, supports compatible evolution, and supplies serializers and deserializers. section 3.2
Serialization separates application objects from Kafka’s byte-oriented transport:
- Producers provide serialized bytes; consumers provide deserialization logic. section 3.2
- Applications can use schemas to enforce expected message structure and manage changes. section 3.2
- Custom implementations use Kafka’s Serializer and Deserializer interfaces when Schema Registry is unavailable or unnecessary. section 3.8
Custom object serialization
A Java serializer can return null for null data, convert an object to bytes with Jackson’s objectMapper.writeValueAsBytes(), and wrap failures in SerializationException. The matching deserializer reconstructs the object with objectMapper.readValue(). section 3.8
Listing 3.43 Serialize method of a custom serializer
@Override
public byte[] serialize(String topic, T data) {
if (data == null) {
return null;
}
try {
return objectMapper.writeValueAsBytes(data); #1
} catch (JsonProcessingException e) {
throw new SerializationException(e);
}
}
Inspect the custom serialize method that converts an object into a byte array and handles serialization errors. section 3.8
