A publish/subscribe model lets services exchange data asynchronously without depending tightly on one another’s internal data models. Producers publish messages to named topics, while consumers receive and process them independently. section 1.1 section 17.1
Kafka strengthens this pattern by persisting published data. Consumers can read messages multiple times over hours, days, or months, supporting real-time processing as well as later analysis. section 1.1
Kafka organizes the flow through a small set of cooperating parts:
- Topics provide named channels or categories for messages. section 1.6
- Partitions divide topics so processing can run in parallel and scale. section 1.6
- Brokers store, distribute, and retrieve messages within the Kafka cluster. section 1.6
- Replication improves resilience when parts of the system fail, while additional brokers provide horizontal scale. section 1.6
One stream, multiple business uses
A retailer can publish inventory and order events through Kafka. Independent consumers can process those events for inventory management, order processing, analytics, or customer-facing workflows without requiring a shared internal data model. section 1.1 section 1.6
Figure 17.1 We differentiate between service-internal data, that is, data on the inside, and data exchanged between services, that is, data on the outside.
Use the service-internal versus service-external data distinction to see where publish/subscribe fits: Kafka carries durable data exchanged between services rather than data kept inside one service. section 17.1
