Message size is a nonfunctional requirement describing the amount of data carried by each Kafka message. It helps shape producer, broker, and consumer configuration because payload size affects batching, serialization, transmission, processing, memory, and network-buffer requirements. section 10.1.4
Capture message-size requirements before tuning the Kafka architecture:
- Average message size describes the typical on-wire footprint and informs batch size, linger time, and expected compression efficiency. section 10.1.4
- Peak or maximum message size establishes safe broker and consumer limits; messages that exceed them may be rejected or require special handling. section 10.1.4
- Compression can reduce network traffic, retained storage, and effective message size, especially for repetitive or text-heavy JSON and Avro payloads, while adding CPU overhead. section 10.1.4
- Size characteristics also influence partitioning, buffer allocation, memory usage, throughput, and cloud costs for storage and data transfer. section 10.1.4
Sizing for ordinary and exceptional payloads
A Kafka sizing worksheet records both the average event footprint and the largest expected event. The average guides batching, linger time, and compression evaluation; the maximum is checked against broker and consumer limits so oversized events are rejected predictably or given an explicit handling path. section 10.1.4
