Chapter 10: Data Oriented Architecture
The introduction frames data-oriented architecture as an approach in which objects serve as gatekeepers for exchanging data.
After this chapter you will be able to
- Objects as data gatekeepers. Explain that data-oriented architecture still uses objects, but gives them a focused role as gatekeepers for exchanging data.
- Monthly change report ingestion. Describe how the service receives and processes the external team's monthly files, including what the files may contain and how new and existing records must be saved.
- Architecture Starting Constraints. Explain that the architecture must be designed from a general outline and three supplied libraries, without frameworks, ORMs, or auto-wiring, and that a good design should not depend on the libraries’ unspecified details.
- Leaky interfaces and change. Explain why programming to interfaces in this example does not isolate the system from CloudStore and database details.
- Dependency-shaped system boundaries. Explain why the section says this design largely mirrors other systems rather than expressing the program’s own design, and state the architectural direction it draws from that observation.
- Data flow before abstractions. Trace the proposed data flow from a request’s serialized payload through JSON, a Manifest, raw CSV, a domain representation, and database inserts and updates. Explain why this sketch comes before choosing objects or implementation details.
- Task-specific domain representation. Justify choosing an Account, Invoice, and Dispute representation for the current database-ingestion task rather than seeking one canonical representation for every use.
- Deferred abstraction around data. Explain why the author recommends deferring abstraction while studying the data, and distinguish deferring abstraction from avoiding it altogether.
- Three architectural worlds. Describe the program as a central world distinct from the things it uses and the things that use it. Classify a database or cloud store as something the program uses, and a REST API as something that uses the program.
- Boundaries against external pressure. Explain why the central program needs boundaries with the other worlds. Describe how external APIs can gradually shape the program around designs it does not control.
- Outward flow of program semantics. Explain why the central program should push its view of the world outward, while still agreeing on representations that work at its boundaries with other worlds.
- Middle world data ownership. Explain why the middle world should define its own data types without depending on external representations such as CSV or TableRecords, and why other worlds should adapt to that design.
- Objects as data suppliers. Explain why a data-oriented program decides what data it needs before designing the objects that supply it.
- External API boundary. Describe why the program introduces a FileRepository instead of letting its domain work directly with the CloudStore API.
- Deterministic core and external shell. Explain how the FileRepository separates non-deterministic file retrieval from deterministic CSV parsing and data construction, and why that separation makes tests easier.
- File Parsing Boundary. Explain why letting callers specify how files from the object store are parsed damages the architectural boundary.
- Cost of speculative extensibility. Explain why the section favors keeping routine code simple rather than designing for hypothetical future changes. Describe how a new file type might require adding a Repository method and updating its callers, and why the author considers that an acceptable cost.
- Application semantics at boundaries. Explain why an application should express the operation it needs, such as upserting data, instead of adopting a dependency’s insert-and-update vocabulary. Describe how a boundary object can provide that operation when the library cannot.
- Ownership of dependency details. Explain why moving insert-and-update decisions to the external side of the boundary improves the design, even though that code still has to exist.
- Architectural checks and balances. Explain how the different “worlds” in an architecture influence one another, and why a dependency may reject an application’s demands.
- Domain data at the boundary. Explain why the program should demand well-modeled domain types from the technologies that use it rather than accept HTTP or JSON as its own language.
- External adapter and testability. Describe what the demand for well-modeled data implies for the surrounding architecture and for testing the program.
- Atomicity Across Repository Writes. Explain why the existing code can leave a partial state despite its good architectural boundaries.
- Boundary Fitting Solutions. Justify why the atomicity fix should stay in the database-facing world rather than expose database internals to the program’s core. Describe the section’s suggested object-based approach at the level presented.
- Boundary-Enabled Integration Changes. Explain why moving from the failing REST integration to an asynchronous strategy need not require a rewrite of the core program.
- Three Worlds and Data Boundaries. Explain the three architectural worlds and why making their boundaries speak data helps decouple them, from application-level boundaries down to method APIs.
What it builds on
- Hidden knowledge, partial construction, and shotgun validation (chapter 9) · The problem of operating on partial data.
- Telling a story with data (chapter 5) · Using data to tell the story of a process.
- Capturing semantics in code (chapter 3) · Capturing the meaning of data in code.
- “What does it mean?” (chapter 3) · Asking what domain data means.
- Separate how you get the data from what you do with it (chapter 5) · Separates how data is obtained from what the program does with it.
- Draw boundaries with data (chapter 9) · Drawing boundaries with data.
- Organizing code around determinism (chapter 6) · Organizing code around determinism
Section by section
Introduction
The introduction frames data-oriented architecture as an approach in which objects serve as gatekeepers for exchanging data.
- Objects as data gatekeepers: Explain that data-oriented architecture still uses objects, but gives them a focused role as gatekeepers for exchanging data.
Why it matters: This frames the role of objects before the chapter develops its architectural approach.
Key terms: data-oriented architecture, objects, gatekeepers, exchanging data
10.1 A small example
Introduces a file-ingestion example in which a service processes monthly change reports from an external team.
- Monthly change report ingestion: Describe how the service receives and processes the external team's monthly files, including what the files may contain and how new and existing records must be saved.
Why it matters: The difference between new, updated, and absent records defines what this ingestion example must handle.
Key terms: file store, manifest, Accounts, Invoices, Disputes
10.1.1 Starting from scratch
The section establishes the minimal external libraries available for a from-scratch architecture and leaves their specifics deliberately vague.
- Architecture Starting Constraints: Explain that the architecture must be designed from a general outline and three supplied libraries, without frameworks, ORMs, or auto-wiring, and that a good design should not depend on the libraries’ unspecified details.
Why it matters: This establishes what the architecture must supply rather than assume is already provided. - Supplied Library Interfaces: Describe the limited capabilities supplied by the database, cloud-storage, and REST libraries: database records and transactional changes, cloud objects that yield bytes, and Request and Response objects for the API.
Why it matters: These are the concrete external capabilities the proposed architecture can start from.
Key terms: architecture, frameworks, ORMs, auto-wiring, TableRecord, inserts, updates, getObject(), CloudObject, read()
10.2 The Object Oriented Thought Process
The section follows an object-oriented design for file ingestion, then shows how its interfaces can still leak dependency details, permit unsafe partial processing, and obscure the system’s own boundaries.
- Behavior-first interface design: Describe how the object-oriented approach starts with actions—downloading, transforming, and saving—and turns them into interfaces with interchangeable implementations.
Why it matters: This explains both the appeal of the design and the starting point for the section’s critique. - Leaky interfaces and change: Explain why programming to interfaces in this example does not isolate the system from CloudStore and database details.
Why it matters: An interface is not a useful boundary if changes behind it still spread through the system. - Partial ingestion risk: Explain how processing files before checking them all can leave the system in an unsafe state when ingestion fails.
Why it matters: The design has consequences for correctness, not just maintainability. - Dependency-shaped system boundaries: Explain why the section says this design largely mirrors other systems rather than expressing the program’s own design, and state the architectural direction it draws from that observation.
Why it matters: It gives the reason to reconsider where the system’s boundaries come from, rather than merely adding more interfaces.
Key terms: interface, Strategy Pattern, Transformer, BulkLoader, Factory, tightly coupled, open to extension, vulnerable to change, CloudStore, partial data
10.3 The Data Oriented Thought Process
The section sketches an ingestion system as a flow of data and argues for shaping its domain data around the task rather than around external formats or a universal model.
- Data flow before abstractions: Trace the proposed data flow from a request’s serialized payload through JSON, a Manifest, raw CSV, a domain representation, and database inserts and updates. Explain why this sketch comes before choosing objects or implementation details.
Why it matters: Sketching the data first lets you see the whole task before implementation choices constrain the design. - External formats and meaning: Distinguish the serialized and CSV representations from data types that express what the input means to this program.
Why it matters: An input format need not dictate the data model used by your program. - Task-specific domain representation: Justify choosing an Account, Invoice, and Dispute representation for the current database-ingestion task rather than seeking one canonical representation for every use.
Why it matters: A model designed for the work at hand avoids inheriting unsuitable external shapes or serving incompatible uses.
Key terms: Data Orientation, ascii arrow (->), Request, Manifest, RawCsvRepresentation, DomainRepresentation, JSON, Account, Invoice, Dispute
10.3.1 Resist the urge to abstract!
This subsection argues for delaying abstractions so the design can be guided by the data rather than by premature groupings of functionality.
- Deferred abstraction around data: Explain why the author recommends deferring abstraction while studying the data, and distinguish deferring abstraction from avoiding it altogether.
Why it matters: Delaying a tidy-looking abstraction can help you notice more important questions about the data.
Key terms: abstract, data, operational baggage
10.4 A way of thinking about software architecture
This section replaces rigid architecture tiers with a three-world model centered on a program’s own data and semantics.
- Three architectural worlds: Describe the program as a central world distinct from the things it uses and the things that use it. Classify a database or cloud store as something the program uses, and a REST API as something that uses the program.
Why it matters: This gives you a simpler way to place architectural boundaries without debating which tier a dependency belongs to. - Boundaries against external pressure: Explain why the central program needs boundaries with the other worlds. Describe how external APIs can gradually shape the program around designs it does not control.
Why it matters: Recognizing this pressure helps you keep an external system’s design from quietly becoming your program’s design. - Outward flow of program semantics: Explain why the central program should push its view of the world outward, while still agreeing on representations that work at its boundaries with other worlds.
Why it matters: This preserves the program’s intended meaning while allowing it to work with systems outside its control.
Key terms: three distinct and broad amorphous “worlds”, things we use, things that use us, boundaries, corruptive pressure, public APIs, middle world, semantics, checks and balances
10.4.1 Start in the middle and work outward
The section argues that architecture should begin with the program’s own data types and make external systems adapt to them.
- Middle world data ownership: Explain why the middle world should define its own data types without depending on external representations such as CSV or TableRecords, and why other worlds should adapt to that design.
Why it matters: This gives you a way to decide which side of a boundary should adapt when data representations differ. - Objects meeting design demands: Distinguish the role assigned to objects here from the traditional object-oriented focus on what an object offers.
Why it matters: It clarifies that starting with data does not mean abandoning objects.
Key terms: middle world, outside worlds, data types, objects, Traditional OOP, Data Oriented Programming
10.4.2 The role of objects in data-oriented programming
This section explains how objects serve a data-oriented design by supplying the domain data the program demands and shielding its core from external APIs.
- Objects as data suppliers: Explain why a data-oriented program decides what data it needs before designing the objects that supply it.
Why it matters: Starting with the data you need keeps available object APIs from setting your program’s terms. - External API boundary: Describe why the program introduces a FileRepository instead of letting its domain work directly with the CloudStore API.
Why it matters: The boundary lets the application keep its own domain language even when a dependency uses different types. - Isolation from external change: Explain how the boundary protects the core program from changes in external storage and representation.
Why it matters: Keeping storage details outside the core makes changes to those details less likely to affect domain code.
Key terms: data, objects, vend data, demands, FileRepository, CloudStore, Manifest, ResourcePath, InputStream, architectural gateway
10.4.3 Designing objects with functional core; imperative shell
This section shows how to organize a data-oriented object around determinism, separating external I/O from logic that can be tested in isolation.
- Deterministic core and external shell: Explain how the FileRepository separates non-deterministic file retrieval from deterministic CSV parsing and data construction, and why that separation makes tests easier.
Why it matters: This boundary makes the object easier to reason about and its logic easier to test.
Key terms: determinism, functional core, imperative shell, FileRepository
10.4.4 Keep resisting the urge to abstract!
The section warns that an apparently flexible file-parsing abstraction can reverse architectural responsibilities, while arguing for cautious abstraction primarily over data.
- File Parsing Boundary: Explain why letting callers specify how files from the object store are parsed damages the architectural boundary.
Why it matters: An abstraction can make an API look more flexible while putting knowledge on the wrong side of a boundary. - Cautious Data Abstraction: Distinguish the section’s warning against early abstraction from a blanket rejection of abstractions.
Why it matters: The lesson is to choose what to abstract, not to avoid abstraction altogether.
Key terms: architectural boundary, middle world, object store, parser, early abstraction, abstracting over data, real-world processes
10.4.5 But what if it changes?
This section argues that, for routine business code, making a small change when it is needed can be cheaper than building abstractions for changes that may never happen.
- Cost of speculative extensibility: Explain why the section favors keeping routine code simple rather than designing for hypothetical future changes. Describe how a new file type might require adding a Repository method and updating its callers, and why the author considers that an acceptable cost.
Why it matters: It helps you weigh the real cost of a change against the ongoing cost of abstractions built for changes that may never happen.
Key terms: Repository, extension, indirection, You Ain’t Gunna Need It
10.4.6 Push your semantics into the external worlds
The section shows how to protect a program’s meaning from external APIs by putting the operations it needs at the boundary rather than spreading API workarounds through the program.
- Application semantics at boundaries: Explain why an application should express the operation it needs, such as upserting data, instead of adopting a dependency’s insert-and-update vocabulary. Describe how a boundary object can provide that operation when the library cannot.
Why it matters: The program can say what it wants done without being shaped by a library’s limited API. - Ownership of dependency details: Explain why moving insert-and-update decisions to the external side of the boundary improves the design, even though that code still has to exist.
Why it matters: Keeping dependency details outside the program limits the effect of a library change. - Centralized approval retrieval: Describe how the approval-service example illustrates the cost of placing missing API functionality throughout the application instead of at one boundary.
Why it matters: A small API mismatch can become expensive when its workaround is repeated across the codebase.
Key terms: semantics, architectural boundary, upsert, Inserts, Updates, boundary, shift in ownership, approvals, boundary violation, sequential service calls
10.4.7 Checks and balances
The section explains why data-oriented architecture needs a push and pull between an application and its dependencies, rather than letting the application’s preferred semantics dictate every interface.
- Architectural checks and balances: Explain how the different “worlds” in an architecture influence one another, and why a dependency may reject an application’s demands.
Why it matters: A useful boundary lets both sides contribute what they know about the design. - Database constraints on design: Describe how the database example shows a dependency’s knowledge influencing application design, and why complete technology agnosticism is not an absolute rule.
Why it matters: The database example makes clear why ignoring a dependency’s real constraints can lead to a poor design.
Key terms: three worlds, push/pull, semantics, outside worlds, Repository, Data Access Object, scanning, dependencies, agnostic
10.4.8 Dealing with the outside world that uses our program
The section shows how requiring domain data at the program’s boundary keeps HTTP and JSON concerns outside the program.
- Domain data at the boundary: Explain why the program should demand well-modeled domain types from the technologies that use it rather than accept HTTP or JSON as its own language.
Why it matters: This keeps the program’s meaning from being defined by the technology used to access it. - External adapter and testability: Describe what the demand for well-modeled data implies for the surrounding architecture and for testing the program.
Why it matters: The boundary gives external formats a place to live while leaving the program straightforward to test.
Key terms: boundary, domain types, semantics, HTTP, JSON, well modeled data, object
10.4.9 Always watch your boundaries
This section shows how fixing a database atomicity problem can undermine architectural boundaries, and argues for solving the problem within the database-facing part of the program.
- Atomicity Across Repository Writes: Explain why the existing code can leave a partial state despite its good architectural boundaries.
Why it matters: Good boundaries alone do not guarantee that a multi-write operation is safe. - Boundary Fitting Solutions: Justify why the atomicity fix should stay in the database-facing world rather than expose database internals to the program’s core. Describe the section’s suggested object-based approach at the level presented.
Why it matters: A fix should preserve the boundary that keeps database details out of the program’s core.
Key terms: atomicity, transact method, repositories, partial state, architectural boundaries, UpdateAndInserts, outside world
10.5 Good boundaries make change easy
The section uses a failing REST integration to show how good boundaries let an application change its entry point without rewriting its core.
- Boundary-Enabled Integration Changes: Explain why moving from the failing REST integration to an asynchronous strategy need not require a rewrite of the core program.
Why it matters: A well-placed boundary lets you replace a strained integration without a sweeping refactor.
Key terms: three-worlds perspective, integrations, core program, new front door, good boundaries
10.6 Wrapping up
The section frames architecture as boundaries between three worlds and argues that making those boundaries speak data helps decouple them.
- Three Worlds and Data Boundaries: Explain the three architectural worlds and why making their boundaries speak data helps decouple them, from application-level boundaries down to method APIs.
Why it matters: This gives you a simple way to reason about boundaries at any scale.
Key terms: middle, things we use, things that use us, boundaries, data, decouple
10.7 Summary
Where this is used later
- Data orientation makes tests better (chapter 11) · Introduces the functional-core and imperative-shell division.
- Use data to show what’s under test (chapter 11) · Tests can express inputs as data rather than depend on an external interface.
Concepts explained in this chapter
- data-oriented programmingData-oriented programming is a programming approach that organizes software around explicit data representations and operations on those representations.
- dataData is information represented independently of the behaviors that operate on it.
- wrapperA wrapper is a type or software component that encloses another representation or interface and exposes a deliberately shaped semantic interface around it.
- data typeA data type is a classification of data that determines its representation, permitted values, and valid operations.
- insertAn insert is a database write description representing the creation of a new record.
- updateAn update is a database write description representing changes to an existing record.
- semanticsSemantics are the intended meanings that a program expresses through its data and operations.
- factoryA factory function is a function that constructs and returns an instance or structured test datum through a defined parameter interface.
- object-oriented programmingObject-oriented design is a software design approach that organizes functionality around objects, their interfaces, and collaborations.
- semantic couplingSemantic coupling is a dependency in which software structure reflects the meanings, representations, or contracts of external components.
- strategy patternThe Strategy pattern encapsulates interchangeable implementations of an operation behind a common interface.
- repository patternA repository is an abstraction that provides collection-like access to domain objects while hiding the underlying data storage mechanism.
- dependencyA dependency is a relationship in which one software element relies on another element's data, behavior, or availability.
- functional coreThe deterministic core is the central part of an application containing domain logic whose results do not depend on side effects or nondeterministic interactions.
- abstractionAn abstraction is a simplified representation that hides selected implementation details behind a more general concept or interface.
- determinismA deterministic function is a computation whose output is completely determined by its inputs and remains consistent for repeated evaluations with those inputs.
- imperative shellAn architectural pattern that isolates deterministic computation in a functional core and places side effects and nondeterministic interactions in an imperative shell.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
