Chapter 13: Using the CQRS pattern to implement system queries
The introduction explains why some queries need a query-specific CQRS view instead of querying data-owning services, and identifies the resulting tradeoff.
After this chapter you will be able to
- Limits of querying owning services. Explain why querying the services that own the data can be inefficient or fragile for some distributed queries, and why a single service’s database may also be unsuitable for its queries.
- Query-specific CQRS view. Describe how CQRS serves a query from a separate, query-specific view database and how that view stays up to date.
- CQRS view tradeoff. Explain the benefit of separating a query’s view from the services’ databases and the costs that separation introduces.
- Provider query limitations. Explain why the providers cannot individually apply the filters, ordering, and pagination needed for findOrderHistory().
- In-memory join cost. Trace the in-memory-join approach and explain why it can require transferring a large volume of data.
- Page-first fetch limitations. Explain when fetching an Order Service page and then fetching related data by ID is practical, and why it fails for fulfillment-status filtering.
- Single-service query difficulty. Explain why a query need not span services to be difficult, using the Restaurant Service’s findAvailableRestaurants() query as an example.
- Event-updated query database. Explain that a CQRS query reads from a separate, query-optimized database instead of querying the services that own the data, and that published service events keep that database up to date.
- Selective CQRS use. Describe the two motivations for CQRS in microservices—complex or high-volume queries within one service and queries spanning services whose data cannot be joined at query time—and explain why it is applied one query at a time rather than to every query.
- CQRS view participants. Describe the provider services and the view subdomain, including the roles of its database, event handlers, repository, and query handlers. Identify the provider databases and view database as the write and read models.
- Provider event update path. Trace how a change to a provider service’s entity reaches the view database through a provider event, view event handler, and view repository.
- Precomputed view query path. Trace how a client query is answered from the view database, and explain why the Order History example can query already joined data instead of querying provider services at request time.
- Efficient distributed queries. Explain why a CQRS view can serve some queries across multiple services more efficiently than API Composition.
- Reduced query runtime coupling. Explain why a system query served by a CQRS view service has less runtime coupling to provider services than API Composition.
- Query-specific databases and scaling. Describe how separating service and view databases lets CQRS support diverse queries efficiently and scale different workloads independently.
- Event and view development costs. Explain why CQRS adds development work for provider and view services, including event contracts, reliable publishing, view updates, and coordination between teams.
- Infrastructure and operational burden. Describe the extra infrastructure CQRS needs and the ongoing work and risks of operating it.
- Replication lag and client responses. Explain why a client might not read its own write from a CQRS view and describe the client-side responses presented here.
- Event contract design coupling. Explain why CQRS can tightly couple provider and view design even though views do not access provider databases directly.
- CQRS adoption tradeoff. Justify the section’s recommendation to prefer API Composition where possible, reserving CQRS for queries that need it and considering service-boundary changes.
- Scale sensitive view design. Explain how sorting, pagination, data size, and query volume affect the effort put into a view, and describe why efficient database queries and updates are the central design problem.
- Existing versus new views. Justify when to enhance an existing CQRS view and when to create a new one, including the main benefit and cost of each choice.
- Query-specific database and schema choice. Explain why a CQRS view can use a database chosen for its particular query, and why database and schema design must be decided together.
- View record shape and indexes. Describe how a view’s data model and indexes must support both retrieving results and locating records to update.
- Efficient query operation support. Explain why filtering, sorting, and pagination capabilities carry particular weight when selecting a view database.
- Efficient and safe event updates. Describe the CUD capabilities needed for event handlers to locate and change view records efficiently without losing concurrent changes.
- Per-event transaction requirements. Explain why a CQRS view often needs single-record atomicity rather than general multi-record transactions, and identify the atomic work needed for idempotency.
- Query and update driven schema. Explain why a CQRS view schema is shaped by its queries and event-triggered CUD operations rather than by provider data models. Describe the three kinds of data it must store.
- Related data grouping tradeoffs. Describe when related entities or duplicated values can be grouped into view records, and explain the query benefits, storage limits, and consequences for event-handler updates.
- Incrementally maintained analytical aggregates. Explain when an analytical CQRS view should precompute an aggregate and why it stores updateable components such as a sum and count rather than only an average.
- Query and handler indexes. Explain how indexes support finding and ordering query results, as well as locating records for event handlers whose events lack the records' primary keys.
- Repository API and find methods. Describe how the view repository hides the database from event and query handlers, and explain how its find methods implement system queries.
- Concurrent update protection. Explain when multiple event handlers can update one view record concurrently and how the repository prevents lost updates.
- Cross-channel event ordering. Explain why events from different channels can reach a view out of order and how upserts let CUD methods handle either arrival order.
- Duplicate event handling. Distinguish idempotent from non-idempotent CUD methods and describe how the repository can reliably discard duplicate events.
- Eventually consistent query results. Describe how queries can help clients cope with replication lag and how find methods can handle incomplete records caused by out-of-order events.
- View-driven event contents. Explain how to work backward from a CQRS view to decide what data each provider event must contain.
- Aggregate and related IDs. Describe which IDs provider events must carry and what the view uses them for.
- Handler and repository responsibilities. Trace what an event handler does when a provider event arrives and what it delegates to the view repository.
- CQRS view service placement. Explain when a CQRS view should be a standalone service rather than part of an existing service, using scaling needs and team ownership as the deciding criteria.
- Full event replay limitations. Explain why replaying every event since application launch is an impractical way to initialize or rebuild a CQRS view in production.
- Snapshot based view initialization. Describe how provider-published snapshot events initialize a CQRS view and why subsequent state change events are still consumed.
What it builds on
- Overview of the API Composition pattern (chapter 12) · The API Composition pattern for distributed queries.
- The findOrder() system query (chapter 12) · The findOrder() query used for comparison
- Overview of messaging (chapter 8) · The basic idea of services communicating through messages.
- Drawbacks (chapter 12) · The drawbacks of API Composition.
- Designing an asynchronous messaging architecture (chapter 8) · Events can be delivered through messaging channels.
- Design-time coupling (chapter 4) · Design-time coupling between components matters.
- Using the API composition pattern (chapter 12) · API Composition is another pattern for implementing system queries.
- Pattern: Idempotent consumer pattern (chapter 8) · The idempotent consumer pattern for duplicate messages.
Section by section
Introduction
The introduction explains why some queries need a query-specific CQRS view instead of querying data-owning services, and identifies the resulting tradeoff.
- Limits of querying owning services: Explain why querying the services that own the data can be inefficient or fragile for some distributed queries, and why a single service’s database may also be unsuitable for its queries.
Why it matters: These limitations explain why a query may need an alternative to reading directly from the data-owning services. - Query-specific CQRS view: Describe how CQRS serves a query from a separate, query-specific view database and how that view stays up to date.
Why it matters: A query-specific view avoids fetching and combining provider data when the query runs. - CQRS view tradeoff: Explain the benefit of separating a query’s view from the services’ databases and the costs that separation introduces.
Why it matters: The faster, better-suited read path is not free: it trades immediate consistency and simplicity for a separate view.
Key terms: API Composition, API composer, runtime coupled, commands, CQRS pattern, view database, provider services, events, eventually consistent, view
13.1 When API Composition isn’t the solution
This section introduces two query problems that API Composition cannot solve efficiently: distributed queries and single-service queries limited by the service’s database.
13.1.1 Distributed queries that can’t use the API Composition pattern
The section uses a consumer’s order-history query to show why API composition becomes inefficient when filtering, sorting, and pagination depend on data spread across services.
- Order history query requirements: Describe what findOrderHistory() returns and why answering it requires data from multiple services.
Why it matters: These requirements explain why retrieving several orders is harder than retrieving one. - Provider query limitations: Explain why the providers cannot individually apply the filters, ordering, and pagination needed for findOrderHistory().
Why it matters: The composer cannot simply ask every service for its matching page and combine the answers. - In-memory join cost: Trace the in-memory-join approach and explain why it can require transferring a large volume of data.
Why it matters: Doing the final query work in the composer can mean fetching far more data than appears on the requested page. - Page-first fetch limitations: Explain when fetching an Order Service page and then fetching related data by ID is practical, and why it fails for fulfillment-status filtering.
Why it matters: The apparent shortcut can still be inefficient or yield the wrong page, leaving the composer to recreate database-like query processing.
Key terms: findOrderHistory(), OrderHistory, fulfillment status, orderCreationDate, provider, API composer, in-memory join, cutoffDate, Order Service, bulk fetch API
13.1.2 Challenging single-service queries
This section explains why queries confined to one service can still be difficult to support with that service’s primary database.
- Single-service query difficulty: Explain why a query need not span services to be difficult, using the Restaurant Service’s findAvailableRestaurants() query as an example.
Why it matters: It shows why keeping all the data in one service does not necessarily make its queries easy to serve. - Indexing and write performance: Explain the tradeoff between adding indexes for faster queries and maintaining write performance.
Why it matters: A faster read path can impose a cost on the same database’s commands. - Denormalized query schema tradeoff: Explain why expensive joins can motivate a denormalized schema and what maintaining that schema costs.
Why it matters: A query shortcut can make the service’s update logic harder to maintain. - Non-relational query requirements: Describe why some single-service queries may need capabilities beyond those of the service’s primary database, without assuming relational databases have none of those capabilities.
Why it matters: The right database choice depends on the query’s actual capabilities, not simply on whether the data belongs to one service. - High-volume query scaling: Explain the limits of serving heavy query traffic from a service’s primary database and the qualified role of read replicas.
Why it matters: Adding capacity for reads does not necessarily remove the service’s query-design burden.
Key terms: Restaurant Service, findAvailableRestaurants(), hybrid query, CQRS pattern, indexes, read performance, write throughput, denormalized schema, pre-joins, normalized and denormalized representations
13.2 Using the CQRS pattern
The section introduces CQRS as an event-updated, query-optimized read model and explains when its separation of reads and writes is useful.
- Event-updated query database: Explain that a CQRS query reads from a separate, query-optimized database instead of querying the services that own the data, and that published service events keep that database up to date.
Why it matters: This is the central alternative CQRS offers to querying the data-owning services directly. - Selective CQRS use: Describe the two motivations for CQRS in microservices—complex or high-volume queries within one service and queries spanning services whose data cannot be joined at query time—and explain why it is applied one query at a time rather than to every query.
Why it matters: Knowing when not to use CQRS is part of choosing the right way to implement a query. - From CQS to CQRS: Distinguish CQS’s rule for individual methods from CQRS’s architectural separation of read and write models, and state why this separation became useful for asymmetric read and write demands.
Why it matters: The distinction clarifies what CQRS separates and how it differs from the older method-level rule.
Key terms: CQRS, query-optimized database, events, API Composition, single service, multiple services, Command-Query Separation, CQS, Command-Query Responsibility Segregation, read and write models
13.2.1 Overview of the CQRS pattern
The section explains the participants in a CQRS view and how provider events keep its database ready to answer system queries.
- CQRS view participants: Describe the provider services and the view subdomain, including the roles of its database, event handlers, repository, and query handlers. Identify the provider databases and view database as the write and read models.
Why it matters: These roles make the two paths through the CQRS pattern easier to follow. - Provider event update path: Trace how a change to a provider service’s entity reaches the view database through a provider event, view event handler, and view repository.
Why it matters: The event path explains how the read-side data is maintained before a query arrives. - Precomputed view query path: Trace how a client query is answered from the view database, and explain why the Order History example can query already joined data instead of querying provider services at request time.
Why it matters: It explains why CQRS can answer this query without composing responses from provider services when the client calls. - View database design priorities: Explain the two stated priorities for choosing the view database type and schema: efficient view queries first, and efficient event-driven updates second.
Why it matters: The database must work for both paths, even though serving queries takes priority.
Key terms: provider services, view subdomain, CQRS view, view service, view database, view event handlers, update path, provider event, view event handler, view repository
13.2.2 The benefits of the CQRS pattern
This section explains five benefits of CQRS: efficient distributed queries, reduced runtime coupling, query-specific optimization, simpler responsibilities, and querying over event-sourced data.
- Efficient distributed queries: Explain why a CQRS view can serve some queries across multiple services more efficiently than API Composition.
Why it matters: This explains when a prepared view is a better choice than composing results at query time. - Reduced query runtime coupling: Explain why a system query served by a CQRS view service has less runtime coupling to provider services than API Composition.
Why it matters: Keeping provider calls out of the query path can make the query faster and more available. - Query-specific databases and scaling: Describe how separating service and view databases lets CQRS support diverse queries efficiently and scale different workloads independently.
Why it matters: Different query workloads need not compete for one database design or one scaling decision. - Simpler schemas and responsibilities: Explain how CQRS’s separation of concerns can simplify services, schemas, and team responsibilities.
Why it matters: The benefit is not only query performance; it can also make implementation easier to reason about. - Queries over event-sourced aggregates: Explain why an event sourcing-based application uses CQRS views for queries beyond primary-key lookup.
Why it matters: Views provide the query capability that the event store itself lacks.
Key terms: CQRS view, API Composition pattern, in-memory joins, pre-joins, runtime coupling, view service, provider services, partial failure, service database, view database
13.2.3 The drawbacks of the CQRS pattern
This section explains the development, infrastructure, operational, consistency, and coupling costs of CQRS and how those costs affect when to use it.
- Event and view development costs: Explain why CQRS adds development work for provider and view services, including event contracts, reliable publishing, view updates, and coordination between teams.
Why it matters: A query-optimized view brings responsibilities on both sides of the event stream, not just a new query endpoint. - Infrastructure and operational burden: Describe the extra infrastructure CQRS needs and the ongoing work and risks of operating it.
Why it matters: The cost of a view includes keeping its infrastructure running and detecting incorrect data, not merely provisioning it. - Replication lag and client responses: Explain why a client might not read its own write from a CQRS view and describe the client-side responses presented here.
Why it matters: Clients need a way to avoid showing users an apparently lost update. - Cross-channel event ordering: Explain how events from multiple channels can produce temporarily inconsistent CQRS query results.
Why it matters: Even apart from ordinary replication lag, combining event streams can expose surprising query results. - Event contract design coupling: Explain why CQRS can tightly couple provider and view design even though views do not access provider databases directly.
Why it matters: Avoiding direct database access does not eliminate dependencies between services. - CQRS adoption tradeoff: Justify the section’s recommendation to prefer API Composition where possible, reserving CQRS for queries that need it and considering service-boundary changes.
Why it matters: The section’s drawbacks are criteria for choosing a query design, not costs to accept automatically.
Key terms: provider services, view services, event contracts, transactional outbox, idempotency, message broker, view databases, consumer lag, end-to-end latency, replication lag
13.3 Designing a CQRS view for a system query
This section explains how query requirements and scale shape CQRS view design, then weighs enhancing an existing view against creating a new one.
- System query kinds: Distinguish identity, relationship, predicate, and analytical queries by what each retrieves or computes.
Why it matters: Recognizing the query kind clarifies what the view needs to return. - Scale sensitive view design: Explain how sorting, pagination, data size, and query volume affect the effort put into a view, and describe why efficient database queries and updates are the central design problem.
Why it matters: It prevents needless view complexity while keeping demanding queries efficient enough for their SLOs. - Existing versus new views: Justify when to enhance an existing CQRS view and when to create a new one, including the main benefit and cost of each choice.
Why it matters: This choice balances the cost of another view against the complexity of making one view serve unrelated queries.
Key terms: identity query, relationship query, predicate query, analytical query, view schema, view repository, normalized schema, denormalizing, indexing, precomputing
13.3.1 Selecting the view database
Explains how to select a CQRS view database and schema by matching the view’s query and event-driven update requirements to database features.
- Query-specific database and schema choice: Explain why a CQRS view can use a database chosen for its particular query, and why database and schema design must be decided together.
Why it matters: Choosing the product without testing a schema against the view’s reads and updates can leave the view inefficient. - View record shape and indexes: Describe how a view’s data model and indexes must support both retrieving results and locating records to update.
Why it matters: A useful record shape is not enough if queries or event handlers cannot find those records efficiently. - Efficient query operation support: Explain why filtering, sorting, and pagination capabilities carry particular weight when selecting a view database.
Why it matters: A database that can express a query may still be a poor choice if executing it is too costly. - Efficient and safe event updates: Describe the CUD capabilities needed for event handlers to locate and change view records efficiently without losing concurrent changes.
Why it matters: Fast queries are of little use if events cannot maintain the view promptly and safely. - Per-event transaction requirements: Explain why a CQRS view often needs single-record atomicity rather than general multi-record transactions, and identify the atomic work needed for idempotency.
Why it matters: The view’s limited transaction needs widen the range of databases that can serve it safely. - Operational and product fit: Explain how load, availability, and specific database features affect the final choice beyond a database’s broad type.
Why it matters: A feature match on paper does not guarantee the view can handle its real workload or remain available.
Key terms: CQRS view, view database, schema, data model, record, secondary indexes, application-maintained index, query operations, filtering, sorting
13.3.2 Designing the view schema
This section explains how to design a CQRS view schema so queries are fast while event handlers can efficiently update the view.
- Query and update driven schema: Explain why a CQRS view schema is shaped by its queries and event-triggered CUD operations rather than by provider data models. Describe the three kinds of data it must store.
Why it matters: A view that answers queries quickly still needs to be practical to keep up to date. - Related data grouping tradeoffs: Describe when related entities or duplicated values can be grouped into view records, and explain the query benefits, storage limits, and consequences for event-handler updates.
Why it matters: Fewer reads are useful only if the resulting records remain workable to store, search, and update. - Incrementally maintained analytical aggregates: Explain when an analytical CQRS view should precompute an aggregate and why it stores updateable components such as a sum and count rather than only an average.
Why it matters: Doing aggregation as events arrive avoids repeating an expensive scan for every analytical query. - Event derived similarity vectors: Trace how a similarity-query view gets its stored vectors and how a query uses them.
Why it matters: Similarity search needs data prepared when view records are written, not just fields copied from providers. - Query and handler indexes: Explain how indexes support finding and ordering query results, as well as locating records for event handlers whose events lack the records' primary keys.
Why it matters: The right indexes prevent expensive query scans and let event handlers find the records they need to change.
Key terms: CQRS view, CUD operations, provider entities, grouping, denormalize, records, event handler, precompute, aggregate, sum and count
13.3.3 Designing the view repository
This section explains how a CQRS view repository serves queries and applies events while handling concurrent, out-of-order, and duplicate updates and the view’s eventual consistency.
- Repository API and find methods: Describe how the view repository hides the database from event and query handlers, and explain how its find methods implement system queries.
Why it matters: This establishes what the repository does for both sides of a CQRS view. - Event-driven record changes: Explain how CUD methods use event properties to change view records and locate records when an event lacks their primary key.
Why it matters: An event cannot update the right view record unless the repository can identify that record. - Concurrent update protection: Explain when multiple event handlers can update one view record concurrently and how the repository prevents lost updates.
Why it matters: Without protection, simultaneous event processing can silently discard a change to the view. - Cross-channel event ordering: Explain why events from different channels can reach a view out of order and how upserts let CUD methods handle either arrival order.
Why it matters: Correct view maintenance cannot rely on the publication order of events carried by different channels. - Duplicate event handling: Distinguish idempotent from non-idempotent CUD methods and describe how the repository can reliably discard duplicate events.
Why it matters: Duplicate delivery must not corrupt the view or unnecessarily move it back to an older state. - Eventually consistent query results: Describe how queries can help clients cope with replication lag and how find methods can handle incomplete records caused by out-of-order events.
Why it matters: These measures let clients recognize a lagging view without pretending that replication lag can be eliminated.
Key terms: view repository, find methods, query handlers, CUD methods, primary key, foreign key, secondary index, concurrent updates, aggregate instance, pessimistic locking
13.3.4 Designing the provider events and their handlers
This section explains how to design the provider events that supply a CQRS view and how event handlers pass those events to the view repository.
- View-driven event contents: Explain how to work backward from a CQRS view to decide what data each provider event must contain.
Why it matters: A view cannot answer its query or stay current if its events omit data it needs. - Aggregate and related IDs: Describe which IDs provider events must carry and what the view uses them for.
Why it matters: The view needs these IDs to maintain the right records and make them findable by later queries. - Cross-team event contracts: Explain why designing provider events requires agreement between teams.
Why it matters: The view’s data requirements have to become event contracts that provider teams can publish. - Handler and repository responsibilities: Trace what an event handler does when a provider event arrives and what it delegates to the view repository.
Why it matters: This division keeps event handlers simple and puts view updates in the repository. - Repository-level duplicate handling: Explain where this section usually places idempotency for provider-event processing and why.
Why it matters: It clarifies which component should normally take responsibility when the same event is processed more than once.
Key terms: provider events, view database, Order History view, domain event, aggregate’s ID, related aggregates, foreign keys, OrderCreated, event contracts, provider services
13.3.5 Deploying and updating the view service
This section explains where to deploy a CQRS view and how to initialize or rebuild one after an application has been running.
- CQRS view service placement: Explain when a CQRS view should be a standalone service rather than part of an existing service, using scaling needs and team ownership as the deciding criteria.
Why it matters: The placement choice determines whether the view can scale and be released independently. - Full event replay limitations: Explain why replaying every event since application launch is an impractical way to initialize or rebuild a CQRS view in production.
Why it matters: These limits explain why a seemingly simple rebuild strategy can fail as an application ages. - Snapshot based view initialization: Describe how provider-published snapshot events initialize a CQRS view and why subsequent state change events are still consumed.
Why it matters: Snapshots provide a practical starting point for a new or rebuilt view without replaying an application’s entire event history.
Key terms: CQRS view, standalone service, provider services, event handlers, message broker, event history, event versions, snapshot event, aggregate, state change events
13.4 Summary
Concepts explained in this chapter
- database indexA database structure that supports efficient retrieval of rows or records based on selected fields or expressions.
- view databaseA view database is a data store dedicated to maintaining and serving the data required by a CQRS view’s queries.
- CQRS viewA query-oriented representation of application data maintained separately from the transactional model in CQRS.
- denormalizationDenormalized data is data stored with deliberate redundancy to make retrieval more efficient.
- view repositoryA software component that applies provider-event data to a query view, including mapping it to a schema and writing it.
- idempotencyThe property of an operation that repeated application of the same input produces the correct effective outcome.
- replication lagA delay between a change being made on one side of a replicated system and becoming visible on another.
- CQRSA distributed query pattern that maintains a read-only queryable replica combining data from multiple services.
- view serviceA view service is a service that maintains and serves a query-oriented view of data.
- upsertA database operation that inserts a record if it is absent and updates it otherwise.
- analytical queryA query that derives summary or analytical results from stored data, often across many records.
- precomputationThe calculation of values in advance so they can be stored and retrieved rather than recomputed for each query.
- event contractAn event contract specifies the data and structure that an event publisher provides to its consumers.
- providerA provider service is a service that owns some of the data returned by a distributed query and exposes that data through an API.
- domain eventAn event representing a business-significant state change of a domain entity or aggregate.
- data groupingThe organization of related data together in a record to match how queries retrieve it.
- schemaA formal definition of the fields, types, and structural rules of a message or data object.
- consumer lagThe number of events a consumer or view has yet to process.
- latencyLatency is the elapsed time between initiating an operation and receiving its result or response.
- tight couplingA relationship in which software elements have substantial dependencies on one another.
- snapshot eventA snapshot event carries a representation of current state so a consumer can initialize or rebuild its data.
- single-service queryA query whose required data is contained within a single service rather than spanning multiple services.
- API composerAn API composer invokes multiple service APIs and combines their results into a client response.
- distributed query executionA data-retrieval operation whose required data resides across multiple services.
- handlerAn event handler is a software component that receives an event and performs the associated processing.
- horizontal scalingA scaling technique that increases capacity by adding concurrent instances of a service or processing component.
- message orderingThe sequencing constraint that messages are processed in the order established by their publication or channel position.
- eventA notification representing something notable that has occurred in an application.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
