liveBook · Manning

Dual read

As explained in ScyllaDB in Action

Dual-read validation is a migration-validation technique that reads corresponding data from source and destination stores and compares the results.

Dual-read validation checks a data migration through the application itself: each request is read from the primary source database and the secondary destination database, then the results are compared. section 12.2.5

A practical validation flow is:

  • Send the normal request to the primary source and also issue it against the destination. section 12.2.5
  • Compare the returned values and treat equivalent results as a successful check. section 12.2.5
  • If values differ, read again because an actively updated row may have changed between the two reads. section 12.2.5
  • Log queries whose results do not converge, then investigate whether the difference is harmless or meaningful. section 12.2.5

Interpreting a mismatch

A mismatch may be functionally equivalent, such as timestamps represented in milliseconds and microseconds, or meaningfully different, such as missing fields or an entire row. User-visible behavior should remain unchanged during the migration. section 12.2.5

Figure 12.8 By dual-reading your application’s request, you can check for mismatches and verify the correctness of your migration.
figure

The figure traces an application request sent to both databases and shows comparison as the validation step. section 12.2.5

How ScyllaDB in Action covers it

How deep it goes
Explains duplicated reads, result comparison, retries, and mismatch logging, but stops short of implementation code.
Examples
Uses timestamp unit differences, missing fields, missing rows, follow-up reads, and logged mismatching queries.
What it assumes
Assumes a source and destination database plus application access to both, while explaining validation from the request level.
What you can do after
Diagnose migration mismatches by comparing live application results and investigating differences that fail to converge.
How it connects
Connects migration validation with dual-write, source and destination databases, Spark Migrator, and two-way scanning.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of ScyllaDB in Action. The section links point to the parts of the book it draws on.