liveBook · Manning

Migration

As explained in Writing for Developers: Blogs that get read

A database migration is the process of changing a database's organization, deployment, or stored data while preserving the application's required behavior.

A database migration changes where data lives, how it is organized, or how its formats represent data while keeping the application usable. Treat it as a transformation and reliability problem, not a file copy: incompatible types, slow tooling, downtime, and verification can dominate the work. section 4.5.1 section 11.3.2

Use this working pattern:

  • Map the source and destination structures, including data types and the target partitioning scheme. section 4.5.1 section 11.3.2
  • Choose a transfer strategy that fits the data volume and transformation cost instead of assuming a direct copy will work. section 4.5.1
  • Reduce downtime with double writes and backfilling, then use integrity checks before relying on the destination. section 11.3.2

Two migrations, two failure modes

Moving roughly 100GiB from legacy Postgres clusters to FakeDB required transforming types that did not translate cleanly; an existing Spark-based tool was estimated to take 21 days. In a different migration, Notion moved from one monolithic database to shards by deciding partition keys and shard counts, then using double writes, backfilling, and integrity checks to avoid prolonged downtime. section 4.5.1 section 11.3.2

How Writing for Developers: Blogs that get read covers it

How deep it goes
Explains type translation, sharding choices, double writes, backfilling, and integrity checks, but stops short of implementation details.
Examples
Uses a Postgres-to-FakeDB transfer and Notion’s migration from a monolithic database to sharded databases.
What it assumes
Assumes familiarity with database deployments, data types, and basic sharding terminology.
What you can do after
Operate/configure staged migrations using data transformation, double writes, backfilling, and integrity checks.
How it connects
Connects database migration with data transformation, sharding, partition keys, double writes, backfilling, and integrity checks.

Also explained in

Compare how 19 books explain migration →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Writing for Developers: Blogs that get read. The section links point to the parts of the book it draws on.