Chapter 11: Architectures that flow
The introduction shows why architecture must be examined as a value stream, even when code and development processes are working well.
After this chapter you will be able to
- Architecture as a value stream. Explain how architectural decisions shape the path a request takes through a running system, including where work flows, waits, or accumulates.
- Limits of local optimization. Explain why clean code and disciplined team practices did not fix the checkout example, and why architectural waste has a wider cost.
- Evidence before architectural commitment. Explain set-based design as exploring several approaches through short, time-boxed, throwaway spikes, then narrowing the options as evidence arrives. Describe how the notification team’s spikes led to a hybrid it might have missed by committing to Redis Pub/Sub immediately.
- Cost and stakes of exploration. Justify when multiple spikes are worth their extra effort: decisions that are difficult or costly to reverse, unfamiliar leading options, or uncertain requirements. Distinguish these from cheap-to-reverse choices, and explain why faster AI-generated prototypes do not reduce the risk of choosing the wrong architecture.
- Last responsible decision point. Describe the last responsible moment as the point when not making a decision would eliminate an important option, and distinguish useful delay before that point from progress-blocking delay after it.
- Architectural decision trigger points. Explain when an architectural decision can wait and how to tell when it must be made. Describe the test of asking what code or contract would break if the decision were not made today.
- Compounding cost of deferral. Explain why some decisions cannot wait for a later trigger point. Describe how delaying an auth, observability, encryption, or error-handling choice makes each new endpoint, service, or module more costly to change.
- Evolving architectural standard work. Describe how early choices about logging, error handling, auth, and encryption establish standard work, and why teams must revisit those standards as the system changes.
- Cross-team deferral costs. Trace how postponing a shared decision can create waste for downstream teams. Explain why correcting that divergence may require ownership and authority across the value stream, not just a technical fix within one team.
- Tracked decision triggers. Explain why deliberately deferred architectural decisions need a recorded trigger, and describe how the internal API example uses the first external partner integration as the point to decide on versioning.
- Decision context during incidents. Distinguish a decision that was wrong when made from one whose original constraints no longer hold, and describe how the log supports that distinction during an incident.
- End-to-end request latency. Explain why a fast individual service can still be part of a slow user experience, and how a system-level value stream map exposes time spent across components and between them.
- Synchronous chain redesign. Explain why a strictly sequential service chain increases user-visible latency and how the checkout example separates required ordering from work that can run in parallel or asynchronously.
- System map construction. Describe how to build a system value stream map for one important request path using end-to-end timestamps, per-step durations, and an activity ratio.
- Architectural bottleneck identification. Explain how the food delivery platform's shared database is identified as the constraint from service behavior during peak load.
- Five focusing steps. Trace the five ToC steps through the shared-database example, distinguishing changes that extract existing capacity, reduce demand, and increase capacity.
- Synchronous push failure modes. Explain how synchronous service calls create temporal coupling and how downstream slowness or limited capacity can cause failures, cascading timeouts, and coupled scaling.
- Queued pull at downstream pace. Explain how pull changes the handoff: upstream services leave messages in a queue, while downstream services take work at their own pace; describe what happens during a burst or downstream outage.
- Bounded queue backpressure. Explain how a bounded queue prevents a producer from overwhelming a consumer: when the queue is full, the producer waits until the consumer catches up.
- Visible queue pressure. Describe what the `pressure` metric reveals about queue capacity and how it can prompt alerts, scaling, or producer throttling.
- Hybrid architecture boundaries. Justify retaining synchronous calls where their properties are needed while placing a queue at a boundary where waiting or a constraint calls for pull-based backpressure.
- Takt time and demand. Explain that takt time is the rate at which a system must complete units of work to meet customer demand, and use it to judge whether the architecture can keep up.
- Drum buffer rope rhythm. Describe how the drum, buffer, and rope coordinate work around a system constraint, and relate them to the constraint, queues, and backpressure in this section.
- Customer demand and takt time. Calculate takt time by dividing the available time window by the units customers demand in that window, and explain what the result means for an order-processing system.
- Processing capacity and headroom. Compare processing time with takt time to explain whether a pipeline can meet demand and how much spare capacity it has.
- Live headroom signals. Describe how the section uses live headroom measurements to detect capacity risk and support operational and planning decisions.
- Service budgets and bottlenecks. Apply a system takt budget to sequential services and explain why an oversized stage can determine whether the whole pipeline meets demand.
- Jidoka at dependency boundaries. Explain how a circuit breaker applies jidoka when an external dependency fails, and distinguish stopping the failure from making it visible.
- Poison messages and boundary validation. Trace how a queue handles invalid messages differently from transient failures, and describe the related validation rule at an API boundary.
- Architectural overproduction costs. Explain why services, endpoints, and abstractions built ahead of need are waste even when they work correctly.
- Evidence based architectural expansion. Justify starting with simple, well-bounded structures and expanding them when actual demand warrants the added complexity.
- Connected practices for architectural flow. Explain how a team combines good decision-making, visibility into request flow, protection against overload and defects, and restraint against overproduction.
- Continuous reassessment as systems evolve. Explain why architectural flow requires a repeated cycle of improvement rather than a one-time sequence.
- Request path value stream. Trace a user-important request from entry to response, recording the components it touches and estimating queue, processing, and network time at each step. Identify the largest contributor to non-processing time and propose a change aimed at reducing end-to-end latency by at least 30% without adding services.
- Constraint buffer and rope. Explain how the exercise asks you to identify the request path’s throughput-limiting component, examine what buffers it, and trace what prevents upstream production from exceeding its capacity.
What it builds on
- Understanding flow, from idea to production (chapter 6) · The idea of following value through a development cycle.
- Identifying and avoiding bottlenecks (chapter 6) · The distinction between improving work locally and addressing a bottleneck.
- Gemba and Kaizen in software development (chapter 2) · The Lean practice of going to see what happens rather than relying only on discussion.
- What Kaizen actually means (chapter 10) · Uses the idea of kaizen to improve an existing standard.
- Multiple value streams (chapter 6) · Assumes that work can be viewed across more than one value stream.
- Going beyond symptoms to the root cause (chapter 8) · Looking beyond an incident's symptoms to its root cause.
- Elevating the constraint (chapter 6) · Applies the earlier discussion of a workflow constraint at the architectural level.
- Overproduction (chapter 2) · The Lean idea of overproduction.
Section by section
Introduction
The introduction shows why architecture must be examined as a value stream, even when code and development processes are working well.
- Architecture as a value stream: Explain how architectural decisions shape the path a request takes through a running system, including where work flows, waits, or accumulates.
Why it matters: Looking at the running system reveals delays and waste that a code review or sprint board may not show. - Limits of local optimization: Explain why clean code and disciplined team practices did not fix the checkout example, and why architectural waste has a wider cost.
Why it matters: Improving the team’s visible work may leave the system’s actual constraint untouched.
Key terms: architecture, value stream, queues, bottlenecks, overproduction, architectural waste, constraint, synchronous microservices, bottleneck
11.1 Set-based design
This section explains how short, parallel architecture spikes help teams make consequential design choices using evidence rather than early guesses.
- Evidence before architectural commitment: Explain set-based design as exploring several approaches through short, time-boxed, throwaway spikes, then narrowing the options as evidence arrives. Describe how the notification team’s spikes led to a hybrid it might have missed by committing to Redis Pub/Sub immediately.
Why it matters: A small investment in evidence can prevent an expensive commitment to an approach that fails under real conditions. - Candidate and evaluation criteria: Describe how a team selects spike candidates and frames comparable evaluation questions from its knowledge, benchmarks, operational constraints, requirements, and unacceptable failure modes. Explain the role of asking what could go wrong and what must be true before coding.
Why it matters: Well-chosen questions make the spikes test the constraints and failures that could actually determine the architecture. - Cost and stakes of exploration: Justify when multiple spikes are worth their extra effort: decisions that are difficult or costly to reverse, unfamiliar leading options, or uncertain requirements. Distinguish these from cheap-to-reverse choices, and explain why faster AI-generated prototypes do not reduce the risk of choosing the wrong architecture.
Why it matters: The approach pays off when the cost of a wrong decision outweighs the cost of testing alternatives.
Key terms: set-based design, spikes, throwaway prototypes, go see, genchi genbutsu, candidate set, evaluation questions, operational constraints, throughput, latency
11.2 The last responsible moment
The section applies Just in Time thinking to architecture and defines when an architectural decision can no longer be delayed.
- Early decisions as inventory: Explain why making an architectural decision before it is needed resembles producing inventory too early: it limits options, carries a maintenance cost, and may become waste if requirements change.
Why it matters: This analogy makes the cost of premature architectural commitments visible. - Last responsible decision point: Describe the last responsible moment as the point when not making a decision would eliminate an important option, and distinguish useful delay before that point from progress-blocking delay after it.
Why it matters: It gives you a boundary between learning before a decision and holding up work by postponing it.
Key terms: Just in Time (JIT), inventory, architectural decisions, last responsible moment, important option
11.2.1 Decisions that can wait
This section explains how to defer architectural decisions until code, data, or contracts are about to depend on them, using the monolith-versus-microservices choice as an example.
- Architectural decision trigger points: Explain when an architectural decision can wait and how to tell when it must be made. Describe the test of asking what code or contract would break if the decision were not made today.
Why it matters: This test helps a team avoid committing before it needs to, while still making the decision before dependencies form. - Monolith versus microservices timing: Explain why the section says a team need not choose between a monolith and microservices in its first architecture meeting. Describe what running the system can reveal before that choice becomes necessary.
Why it matters: Waiting for observed behavior can make this particular architecture choice better informed than an initial guess.
Key terms: trigger point, last responsible moment, monolith versus microservices, real traffic patterns
11.2.2 Decisions that cannot wait
The section explains why foundational architecture decisions sometimes need to be made immediately, then revisited as the system and its value stream change.
- Compounding cost of deferral: Explain why some decisions cannot wait for a later trigger point. Describe how delaying an auth, observability, encryption, or error-handling choice makes each new endpoint, service, or module more costly to change.
Why it matters: Making a spreading decision early can prevent repeated retrofits and leave production incidents easier to diagnose. - Evolving architectural standard work: Describe how early choices about logging, error handling, auth, and encryption establish standard work, and why teams must revisit those standards as the system changes.
Why it matters: A shared pattern gives teams a consistent starting point and something concrete to improve. - Cross-team deferral costs: Trace how postponing a shared decision can create waste for downstream teams. Explain why correcting that divergence may require ownership and authority across the value stream, not just a technical fix within one team.
Why it matters: Looking across the value stream reveals costs that the team making a deferred decision may never see.
Key terms: JIT, inventory cost, authentication and authorization model, logging and observability standards, standard work, kaizen cycle, architectural debt, value stream, shared contract, incompatible contracts
11.2.3 Tracking decisions
This section shows how an architectural decision log makes deferred decisions actionable and preserves the context needed to revisit decisions when conditions change.
- Tracked decision triggers: Explain why deliberately deferred architectural decisions need a recorded trigger, and describe how the internal API example uses the first external partner integration as the point to decide on versioning.
Why it matters: A recorded trigger lets a team wait for useful evidence without losing the decision. - Decision context during incidents: Distinguish a decision that was wrong when made from one whose original constraints no longer hold, and describe how the log supports that distinction during an incident.
Why it matters: The recorded context helps an incident review correct an outdated assumption instead of blaming someone for a formerly sound decision.
Key terms: architectural decision log, trigger, deferred, last responsible moment, constraints at the time, root cause analysis, kaizen cycle, superseded
11.3 Value stream mapping at the system level
This section shows how to map a request across an entire architecture to expose hidden wait time, redesign unnecessary blocking, and track architectural flow over time.
- End-to-end request latency: Explain why a fast individual service can still be part of a slow user experience, and how a system-level value stream map exposes time spent across components and between them.
Why it matters: It helps you improve the delays users actually experience rather than optimizing a service that is already fast. - Synchronous chain redesign: Explain why a strictly sequential service chain increases user-visible latency and how the checkout example separates required ordering from work that can run in parallel or asynchronously.
Why it matters: Changing which steps block the response can reduce perceived latency even when a slow external call remains slow. - System map construction: Describe how to build a system value stream map for one important request path using end-to-end timestamps, per-step durations, and an activity ratio.
Why it matters: A measured request path shows where time goes and gives the team a concrete basis for improvement. - Living latency baseline: Explain why the team should maintain its system value stream map as the architecture changes and use its latency breakdown to spot degradation and justify improvements.
Why it matters: A continuing baseline makes emerging delays visible and gives architectural improvements measurable justification.
Key terms: system-level value stream mapping, activity ratio, hidden latency, synchronous chain, parallel, asynchronously, user-visible latency, distributed tracing, queue time, processing time
11.4 Finding the system bottleneck
This section applies the Theory of Constraints to architecture by locating a shared-resource bottleneck and improving system flow through five focusing steps.
- Architectural bottleneck identification: Explain how the food delivery platform's shared database is identified as the constraint from service behavior during peak load.
Why it matters: Comparing affected and unaffected request paths helps you locate the resource that actually limits the system. - Five focusing steps: Trace the five ToC steps through the shared-database example, distinguishing changes that extract existing capacity, reduce demand, and increase capacity.
Why it matters: The sequence favors cheaper ways to restore request flow before costly architectural changes, then prompts you to check what limits flow next. - Constraint focused improvement: Justify why improving a service that is not the constraint may leave overall throughput unchanged.
Why it matters: It helps prevent substantial work on a visible service from being mistaken for an improvement in system flow.
Key terms: Theory of Constraints (ToC), constraint, shared PostgreSQL database, identify, exploit, subordinate, elevate, repeat, flow, system throughput
11.5 Pull versus push
This section contrasts synchronous push architectures with queue-based pull architectures and explains how pull keeps work moving when downstream services are slow or unavailable.
- Synchronous push failure modes: Explain how synchronous service calls create temporal coupling and how downstream slowness or limited capacity can cause failures, cascading timeouts, and coupled scaling.
Why it matters: These failure modes show why service-to-service calls can interrupt flow beyond the service that first has trouble. - Lean costs of pushing work: Describe how the section relates push architecture to Lean problems: keeping services ready without work, imposing work beyond downstream capacity, and allowing failures to propagate.
Why it matters: The Lean analogy makes capacity overload and failure propagation visible as flow problems, not just service errors. - Queued pull at downstream pace: Explain how pull changes the handoff: upstream services leave messages in a queue, while downstream services take work at their own pace; describe what happens during a burst or downstream outage.
Why it matters: A queued handoff lets upstream work continue without requiring every downstream service to be available at once.
Key terms: push, temporal coupling, cascading failures, scaling is coupled, Lean principles, overproduction, backpressure, defect propagating, pull architecture, queue
11.5.1 Backpressure: The safety valve
This section explains how backpressure makes service capacity visible and helps teams choose where to use pull-based queues alongside synchronous calls.
- Bounded queue backpressure: Explain how a bounded queue prevents a producer from overwhelming a consumer: when the queue is full, the producer waits until the consumer catches up.
Why it matters: A bounded queue gives an overloaded service a way to slow incoming work rather than silently drop it or crash. - Visible queue pressure: Describe what the `pressure` metric reveals about queue capacity and how it can prompt alerts, scaling, or producer throttling.
Why it matters: Pressure turns an otherwise hidden capacity constraint into a signal you can act on. - Control of service pace: Distinguish push from pull by identifying whether the producer or consumer controls the pace of a service interaction.
Why it matters: Asking who sets the pace clarifies whether an interaction can slow down when capacity is limited. - Hybrid architecture boundaries: Justify retaining synchronous calls where their properties are needed while placing a queue at a boundary where waiting or a constraint calls for pull-based backpressure.
Why it matters: Choosing boundaries lets you gain backpressure without giving up useful synchronous behavior everywhere.
Key terms: backpressure, bounded queue, producer, consumer, pressure, headroom, bottleneck, push, pull, kanban
11.6 Takt time and system rhythm
The section explains how takt time measures the pace required by customer demand and how drum-buffer-rope coordinates a software system around its constraint.
- Takt time and demand: Explain that takt time is the rate at which a system must complete units of work to meet customer demand, and use it to judge whether the architecture can keep up.
Why it matters: It lets you judge system pace against demand rather than merely observe that the system has a rhythm. - Drum buffer rope rhythm: Describe how the drum, buffer, and rope coordinate work around a system constraint, and relate them to the constraint, queues, and backpressure in this section.
Why it matters: Finding a bottleneck is not enough; the rest of the system needs to work at a pace that the bottleneck can support.
Key terms: takt time, customer demand, system rhythm, drum-buffer-rope, constraint, buffer, rope, queues, backpressure
11.6.1 Calculating takt time
The section calculates takt time from customer demand and uses it to assess capacity, monitor headroom, and find pipeline bottlenecks.
- Customer demand and takt time: Calculate takt time by dividing the available time window by the units customers demand in that window, and explain what the result means for an order-processing system.
Why it matters: Takt time puts architectural performance in terms of the rate customers actually need. - Processing capacity and headroom: Compare processing time with takt time to explain whether a pipeline can meet demand and how much spare capacity it has.
Why it matters: Comparing demand with processing capacity shows whether an architecture needs more concurrency or a faster flow. - Live headroom signals: Describe how the section uses live headroom measurements to detect capacity risk and support operational and planning decisions.
Why it matters: A live demand-relative signal can reveal a shrinking capacity margin before users experience failures. - Service budgets and bottlenecks: Apply a system takt budget to sequential services and explain why an oversized stage can determine whether the whole pipeline meets demand.
Why it matters: A system-wide target is useful only if you can locate the stage that prevents the pipeline from meeting it.
Key terms: takt time, available time window, customer demand, processing time, concurrent processing pipelines, headroom factor, actual throughput, current demand, andon signal, yellow signal
11.7 Building quality in
The section shows how to protect system flow by stopping defects at their source and avoiding architecture that exceeds demonstrated needs.
- Jidoka at dependency boundaries: Explain how a circuit breaker applies jidoka when an external dependency fails, and distinguish stopping the failure from making it visible.
Why it matters: A failing dependency should not turn one problem into a wider system failure. - Poison messages and boundary validation: Trace how a queue handles invalid messages differently from transient failures, and describe the related validation rule at an API boundary.
Why it matters: Rejecting bad input at a boundary prevents one defective message from repeatedly blocking otherwise valid work. - Architectural overproduction costs: Explain why services, endpoints, and abstractions built ahead of need are waste even when they work correctly.
Why it matters: An architecture can impede flow through its upkeep even when no component is broken. - Evidence based architectural expansion: Justify starting with simple, well-bounded structures and expanding them when actual demand warrants the added complexity.
Why it matters: Letting demonstrated needs drive expansion avoids paying for complexity before it provides value.
Key terms: jidoka, andon, circuit breaker, open state, poison message, dead letter queue (DLQ), transient failures, input validation, andon signal, overproduction in architecture
11.8 A system that flows
The section connects the chapter’s architectural practices into a repeatable approach to keeping a system flowing as it changes.
- Connected practices for architectural flow: Explain how a team combines good decision-making, visibility into request flow, protection against overload and defects, and restraint against overproduction.
Why it matters: These practices address different threats to flow and work together rather than standing alone. - Continuous reassessment as systems evolve: Explain why architectural flow requires a repeated cycle of improvement rather than a one-time sequence.
Why it matters: A flowing architecture can lose that quality when the system or its demand changes.
Key terms: set-based design, last responsible moment, value stream mapping, Theory of Constraints, takt time, pull systems, value streams, constraint, kaizen
11.9 Exercise: Mapping, measuring, and improving your architecture
This exercise asks readers to map a critical request path, audit architectural overproduction, design service health responses, and assess the path’s drum-buffer-rope and rhythm.
- Request path value stream: Trace a user-important request from entry to response, recording the components it touches and estimating queue, processing, and network time at each step. Identify the largest contributor to non-processing time and propose a change aimed at reducing end-to-end latency by at least 30% without adding services.
Why it matters: Timing the whole request path helps you target delay that users actually experience. - Architectural overproduction audit: Describe how to audit services, endpoints, and configuration for signs of overproduction, and assess whether services could be merged without losing important properties.
Why it matters: An audit can reveal complexity that sees little use without assuming every separate service is unnecessary. - Critical service andon design: Describe the elements the exercise asks you to design for a critical service: dependency-aware health checks, defined health states, tiered alerts, and protection for a fragile external dependency.
Why it matters: Explicit health signals and response paths make trouble in a critical service easier to detect and act on. - Constraint buffer and rope: Explain how the exercise asks you to identify the request path’s throughput-limiting component, examine what buffers it, and trace what prevents upstream production from exceeding its capacity.
Why it matters: A slow component matters as the drum only if it limits the whole path, and its surrounding flow determines whether it idles or becomes overloaded. - Path rhythm and feedback: Describe how to compare the path’s takt time with constraint throughput, consider ways to close a gap, and make the path’s pace and buffer buildup visible.
Why it matters: Comparing capacity with the needed rhythm and watching buffer buildup makes a developing flow problem visible.
Key terms: value stream, queue/wait time, processing time, network time, activity ratio, constraint, overproduction, average daily traffic volume, API endpoint, configuration parameters
11.10 Summary
Where this is used later
- Lean onboarding (chapter 13) · The decision log referenced as a source for architectural decision records.
- Building a culture of continuous learning (chapter 15) · The architectural decision-log format, here applied to process decisions.
- Andon for production systems (chapter 12) · A visible pressure signal can feed production andon capabilities.
- Gemba in production (chapter 12) · Observing a production system can use the same attention to the path users experience.
Concepts explained in this chapter
- synchronous callProcessing in which an operation waits for current work to complete before proceeding or returning.
- takt timeThe available time per unit of output, calculated by dividing the available production window by demand in that window.
- overproduction wasteLean waste caused by producing more, earlier, or sooner than demand requires.
- last responsible momentA decision-making principle of postponing a decision until the latest point at which it can still be made responsibly, allowing more information to become available.
- backpressureA mechanism by which a downstream system signals that upstream work should be slowed or limited to match its processing capacity.
- bottleneckA stage whose capacity persistently falls short of demand and limits the output of a value stream.
- jidokaA Lean production approach that builds quality into processes by stopping when abnormalities occur and preventing defects from proceeding.
- set-based designAn engineering approach that explores multiple design options in parallel, narrows them as evidence arrives, and delays commitment until further exploration is no longer worthwhile.
- dead letter queueA queue that holds messages or work items that could not be processed successfully through the normal path.
- activity ratioThe proportion of elapsed work time spent on productive activity rather than waiting or other non-progressing periods.
- value stream mappingA Lean technique for representing and analyzing the flow of work through a value stream.
- value streamA value stream is the complete sequence of activities and handoffs through which work produces an outcome for a customer or user.
- constraintA condition or limitation that restricts how work can proceed within a delivery system.
- throughputThe rate at which completed work passes through a production or delivery system.
- theory of constraintsA management discipline that improves system performance by identifying and addressing the constraint that limits throughput.
- AndonA visual management system that signals abnormal conditions so work can be stopped or corrected.
- headroomUnused system capacity reserved to absorb changes or variation in demand without degrading service.
- continuous improvementA feedback loop is a process in which information about outcomes is returned to influence subsequent decisions or actions.
- contractAn application programming interface is a defined set of operations and rules through which software components interact.
- standard workA single prescribed method applied across teams regardless of differences in their work contexts.
- experimentationA deliberately limited activity undertaken to learn about an uncertain idea or approach.
- flowA Lean principle of moving work continuously by reducing queues, delays, and interruptions.
- architecture decision recordAn architecture decision record is a concise document that records an architectural decision, its rationale, and its consequences.
- inventoryInventory is the waste of work items, information, or decisions that have been started or accumulated but are not yet needed downstream.
- retrospectiveA team meeting for reflecting on recent work and identifying improvements.
- YAGNI principleA software design discipline that delays adding structure until a concrete requirement demonstrates the need for it.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
