An event streaming platform captures events from sources such as websites, mobile devices, shipment tracking, and business transactions. It reliably transfers and stores them in occurrence order so other applications can process or analyze them in real time or later. section 1.1
Kafka provides an end-to-end platform around three capabilities: publish and consume records, durable storage, and processing. Brokers serve client requests and store records as bytes; producer clients write records and consumer clients read them independently. section 1.4 section 1.6
A platform-centered event flow separates responsibilities:
- Producers send records to Kafka brokers without needing awareness of consumers. section 1.4
- Brokers provide the storage layer for records and service read and write requests. section 1.6
- Processing applications transform available event data; Kafka Streams supports joins and stateful transformations outside brokers. section 1.6
- Kafka Connect integrates existing systems to move data into and out of Kafka, while Schema Registry supports record compatibility. section 1.6
Retail clickstream and sales processing
A coupon email click produces clickstream data directly into Kafka through a producer microservice. The same events can measure campaign engagement and help attribute purchases, while sales data can be masked, processed for customer points, and exported to a dashboard. section 1.5 section 1.6
Figure 1.3 Using the Kafka event streaming platform with a simplified architecture
Compare point-to-point departmental data connections with a shared Kafka layer, where components write records to Kafka and consumers read without producer awareness. section 1.4
