Batching groups records and processes or transmits them together as one unit. Kafka clients work with record collections: a producer sends one batch in a produce request, and the broker appends that batch to one partition within a topic. section 2.2
KafkaProducer.send() adds records to a batch instead of immediately sending each record. The asynchronous call lets the producer continue adding records while a separate I/O thread transmits the batch when its buffer is full or transmission is due. Fewer requests improve producer and broker performance. section 4.2
A batched produce flow has these roles:
- Create ProducerRecord objects from sales data. section 4.2
- Pass each record to KafkaProducer.send(), allowing the client to accumulate records. section 4.2
- Let the producer I/O thread send the batch and invoke the callback after broker acknowledgment. section 4.2
- Keep callback processing light so it does not hold up sending. section 4.2
Sales records
A sales producer fetches ProductTransaction objects, maps each purchase to a ProducerRecord, and calls producer.send(record, callback). The callback logs the acknowledged offset and timestamp or reports an exception; batching remains inside the producer while the application continues producing. section 4.2
Figure 4.4 The producer batches records and sends them to the broker when the buffer is full or it’s time to send them.
Shows the producer accumulating records and sending them when the buffer is full or transmission time arrives. section 4.2
