A customer should not have to wait for tomorrow’s batch job to see that a product is out of stock. Real-time processing handles events soon after they occur, so systems can update information and react while it is still useful. It contrasts with batch processing, which collects data and produces results at scheduled intervals. section 12.1.2
With Kafka, a producer writes an event to a topic and consumers process it as needed. Kafka persists the event stream, so a consumer that fails can resume processing and a new service can also read older events. Services can therefore exchange data asynchronously without requiring every consumer to be available at the moment an event occurs. section 1.1 section 1.2
An online-store price change
When a product’s price changes, a stream processor can handle that event almost immediately rather than waiting for a scheduled update. An out-of-stock event can likewise prompt a prompt inventory update. These fast responses are valuable, but the current view may be an intermediate result: later events can change it. section 12.1.2
Figure 12.1 n batch processing, a large amount of data is processed at-once at specific times and the results are definite. When processing one event at a time, data is processed almost immediately, but it’s difficult to provide definite results, as the system is eventually consistent.
Compare scheduled batches with event-at-a-time processing: faster updates come with results that may change as further events arrive. section 12.1.2
Real time is not automatically the right choice. Batch jobs can provide definitive results for deadline-based reports and may reduce costs when run overnight. Kafka can feed those jobs too, or support less frequent message processing. Choose the processing schedule according to how quickly a result is needed and whether it must be final. section 12.1.2 section 1.2
