Copy-on-write (COW) materializes updates and deletions at write time by rewriting affected data files. The commit produces new Parquet files and a new snapshot that references them, while older snapshots retain the original files for time travel. section 6.3.1 section 10.3.2
COW changes the balance between write cost, read performance, and retention management:
- Avoids separate delete files, so readers do not reconcile base files with accumulated delete artifacts. section 6.3.1 section 10.1.4
- Increases write amplification and update latency because affected files must be rewritten at commit time. section 6.3.1 section 10.1.4
- Leaves replaced files physically stored until snapshots referencing them expire and cleanup removes unreferenced files. section 10.3.2 section 10.3
- Fits workloads that prioritize predictable reads and simpler retention over minimizing the cost of frequent small updates. section 10.1.4 section 10.3.2
Deleting rows with COW
A delete operation rewrites the affected data file without the deleted rows, records the replacement in a new manifest, and leaves the original file available through the previous snapshot. After snapshot expiration, the unreferenced original file becomes eligible for physical cleanup. section 10.3.2 section 11.2.5
Figure 10.6 In Copy-on-Write, when you delete records, the data file with those records is rewritten without them. In Merge-on-Read, instead of rewriting a whole file, the deleted records are tracked in a delete file or deletion vector. This can be used to reconcile the table when it is read, which enables faster writing to the table but slows down reading.
Compare COW's rewritten data files with merge-on-read's delete files or deletion vectors, including their opposing write and read-performance effects. section 10.1.4
