Apache Kafka is a distributed event streaming platform that lets applications publish records, store them durably, and process them. Brokers accept records from producers, while consumers read them independently, allowing multiple applications to use the same event streams. section 1.4 section 1.6
Figure 1.3 Using the Kafka event streaming platform with a simplified architecture
Use the architecture diagram to see how Kafka separates event producers from consumers and simplifies connections among departments and data sources. section 1.4
Kafka's end-to-end platform combines several cooperating capabilities:
- Producers write records to brokers, and consumer clients read those records without needing awareness of the producers. section 1.4
- Brokers provide durable storage and serve client requests; they store records as bytes without altering their contents. section 1.6
- Schema Registry helps producers and consumers maintain compatible record formats. section 1.6
- Kafka Connect integrates external systems, Kafka Streams transforms data with joins and state, and ksqlDB supports real-time systems with SQL. section 1.6
ZMart clickstream and sales events
A producer microservice sends coupon clicks, purchases, and related activity into Kafka. Marketing measures campaign effectiveness, data science attributes purchases to initial clicks, and downstream applications process customer points or prepare dashboard data. Kafka Connect can also move sales data while masking credit-card numbers. section 1.5
