An order service needs to answer a question immediately: what is the current state of an order? A relational database is well suited to storing that service’s data and querying its current state with SQL. It can execute complex queries across tables and provides ACID guarantees. Kafka serves a different question: what happened, and how did the system reach its current state? section 17.4 section 18.5.1
Keep the boundary between internal data and exchanged data clear:
- Use a relational database for service-specific data, particularly when the service needs SQL queries, joins, or immediate access to current state. section 17.4.1 section 18.5.1
- Avoid making one database the shared interface for many services. Changes then become tightly coupled and may require coordination across teams. section 17.4.1
- Use Kafka for asynchronous data exchange between services. Each service can build the view it needs from the event history; Kafka is not a substitute for a relational database’s synchronous queries or full ACID guarantees. section 17.4.2 section 18.5.1
Sending customer and order data
Suppose customer and order details live in separate, normalized database tables. A consumer may need both in one message. Joining the tables in the relational database before writing a flattened message to Kafka avoids making consumers perform the join—and avoids an expensive, complex join in a real-time pipeline. section 18.5.1
A database also has a log of its own. Its commit log records changes before tables are updated and supports recovery and replication. Kafka makes a log central to data exchange instead; services can turn that history into their own materialized views. Kafka Connect can link databases and Kafka with relatively little configuration effort. section 4.1.1 section 17.4.2
Figure 17.5 Relational databases excel with their consistency guarantees, but when a database is used by many services, this leads to very strong coupling, and even small changes must be coordinated with all teams.
The diagram illustrates the cost of sharing a database across services: even a small database change can require coordination with every dependent team. section 17.4.1
