liveBook · Manning

Data on the inside

As explained in Apache Kafka in Action

Data on the inside is data managed within a service and structured to fit that service's specific needs.

Data on the inside is service-internal data shaped for the specific service that manages it. Other services should not depend on its format. Because users often read and write it directly, it typically favors synchronous operations and strong consistency, so a successful write is immediately visible. section 17.1

The service and its internal data model are tightly coupled. A service may combine and aggregate several subsets of its own data to deliver useful functionality, while optimizing that data model for performance and usability. The trade-off is that changing the model usually requires corresponding service changes. section 17.1

A service’s internal state

An online service can keep data in a model tailored to its own operations, combine several internal subsets, and return a consistent result to users. Relational databases are recommended for this kind of internal data, while Kafka is used for exchanging data between services. section 17.1 section 17.6

Distinguish internal data from data exchanged across service boundaries:

  • Internal data follows the needs of one service; outside data is service-agnostic and structured for exchange. section 17.1
  • Internal data favors synchronous access and high consistency; outside data is ideally immutable because other services may already have consumed it. section 17.1
  • A practical architecture commonly uses relational databases for internal state and Kafka for historical, decoupled data exchange. section 17.6
Figure 17.1 We differentiate between service-internal data, that is, data on the inside, and data exchanged between services, that is, data on the outside.
A red and green paper with black text

AI-generated content may be incorrect.

Use the diagram to contrast service-internal data with service-external data exchanged between services. section 17.1

How Apache Kafka in Action covers it

How deep it goes
Explains storage fit, consistency, aggregation, coupling, and architectural trade-offs, but does not specify an implementation schema.
Examples
Uses user-facing service state, internal aggregation, relational databases, and Kafka-based data exchange as concrete illustrations.
What you can do after
Evaluate whether data belongs in a service’s optimized internal model or in a loosely coupled exchange model.
How it connects
Connects service-internal data with data on the outside, service boundaries, relational databases, Kafka, consistency, and synchronous communication.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.