Serialization converts message keys and values into bytes for transfer, while deserialization converts those bytes back into meaningful objects. Kafka brokers transfer binary data without interpreting its content, so producers and consumers must agree on the format. section 3.1.4
Kafka applications determine serialization through client configuration:
- Configure key.serializer and value.serializer independently; keys and values may use different serializers. section 3.1.4 section 3.2
- Configure matching deserializers on consumers, because incompatible formats produce runtime errors. section 3.1.4
- Choose primitive, JSON, schema-based, or custom implementations according to application and data-contract needs. section 3.1.4 section 3.1.6
Customer 360 producer configuration
A Customer 360 project considers JSON serialization and initially assigns StringSerializer to both keys and values. Its consumers must use corresponding StringDeserializer settings. The key type remains architecturally important: customer IDs preserve per-customer ordering, while transaction IDs help distribute load across partitions. section 3.1.6 section 2.4.2
key.serializer=org.apache.kafka.common.serialization.IntegerSerializer value.serializer=org.apache.kafka.common.serialization.StringSerializer
Use the compact producer configuration to inspect separate key and value serializer assignments. section 3.1.4
Figure 3.7 Producer message-sending workflow in Kafka
Use the producer workflow figure to trace serialization from message creation through batching, partitioning, replication, and acknowledgment. section 3.2
