liveBook · Manning

Snapshot-and-stream workflow

As explained in Grokking Software Architecture

A data migration strategy that combines an initial bulk copy of existing records with ongoing capture and replay of subsequent changes.

Snapshot-and-stream migration moves data from a legacy monolith to a new microservice in two phases: copy existing records, then synchronize later changes until both systems match. section 12.5.2

The workflow pairs:

  • Snapshot: make a one-time bulk copy of historical data, using export tools such as pg_dump or AWS DMS. section 12.5.2
  • Stream: capture and replay changes made after the snapshot begins. Options named include Change Data Capture (CDC) and application-level syncing. section 12.5.2

Billing migration

A new Billing microservice needs historical credit-card data from the monolith. Copy the existing records, then synchronize ongoing changes before switching over. section 12.5.2

Figure 12.4 The Strangler Fig Migration. Instead of a risky "Big Bang" rewrite, an API Gateway incrementally reroutes traffic to the new Microservice, while Change Data Capture (CDC) safely syncs the underlying data in the background.

See how incremental traffic rerouting and background CDC fit into a Strangler Fig migration. section 12.5.2

How Grokking Software Architecture covers it

How deep it goes
Defines the bulk-copy and synchronization phases, and names CDC and application-level syncing without detailing their implementation.
Examples
Uses a Billing microservice migration and names pg_dump and AWS DMS as snapshot export tools.
What you can do after
Apply it by pairing a historical-data copy with ongoing change synchronization during migration.
How it connects
Connects data migration with microservices, the Strangler Fig pattern, and Change Data Capture.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Software Architecture. The section links point to the parts of the book it draws on.