When independent services exchange messages across servers, platforms, or regions, delivery must remain reliable as the system grows. A messaging system moves messages from producers to consumers. It handles data exchanged between services rather than data kept inside one service. section 1.0 section 17.1 section 17.2
A classic messaging system may take responsibility for more than delivery:
- Route a message to a consumer across different locations or organizational boundaries. section 17.2
- Convert incompatible message formats so the connected systems need not change. section 17.2
- Ask producers to slow down when consumers cannot keep up. section 17.2
That convenience has an operational cost. Classic systems commonly delete messages after consumption, making historical data difficult for a new service to obtain. They may also need to track consumers and manage slow or unavailable ones; a poorly configured or overloaded system can affect data exchange more broadly. section 17.2.2
Figure 17.3 A slow consumer can slow down a producer in classic messaging systems.
The slow-consumer diagram illustrates a coupling risk: backpressure in a classic messaging system can make a producer wait for a consumer. section 17.2.2
Kafka makes a different trade-off. Producers place data in logs, and consumers retrieve it later, subject to the cleanup policy. Services decide what to write and read; Kafka does not provide every routing or transformation feature of an enterprise service bus out of the box. Keeping data for longer can let a new service read historical messages without asking producers to send them again. section 17.2.1 section 17.2.2
