A Kafka record, also called a message, is one item of data published to a topic and read by consumers. For an online store, each product-price change can be a separate record. An inventory system can react to new prices while an analysis system reads the same changes independently. section 3.0 section 2.4
A record has a value containing its main payload, a timestamp, and optionally a key and custom headers. Kafka sends records as byte arrays and typically groups them into batches. Records with the same key go to the same partition; for a single producer, their order is guaranteed there. section 3.3 section 1.3
Publish a price change
With Kafka running and the products.prices.changelog topic created, run `echo "coffee pads 10" | kafka-console-producer.sh --topic products.prices.changelog --bootstrap-server localhost:9092`. The console producer sends the input as a record. It prints no success confirmation for this piped command, so check with a consumer rather than expecting an acknowledgment on screen. section 2.2 section 2.0
# Consumer 2 window
$ kafka-console-consumer.sh \
--topic products.prices.changelog \
--from-beginning \
--bootstrap-server localhost:9092
coffee pads 10
coffee pads 11
coffee pads 12
coffee pads 10
The consumer command shows how `--from-beginning` retrieves earlier price records, including the first update, after newer updates have arrived. section 2.4
Choose what each record represents and how long it should remain useful:
- A state contains an object's complete information; a delta contains only a change and needs context or a complete state. section 3.3
- An event describes something that happened; a command instructs another system to act. section 3.3
- Retention removes records by age or size; log compaction uses keys to retain the latest record for each key. section 10.0
Keep records small and machine-readable. Kafka is optimized for many messages of 1 MB or less, not for exchanging PDFs. Put a large file in object storage and publish a record pointing to it, or split a large collection of events into individual event records. section 18.5.3
