A deletion vector is a data-file-associated structure for recording rows treated as deleted without creating a separate delete file for each operation. Iceberg v3 permits at most one deletion vector per data file, limiting the unbounded accumulation associated with older delete-file patterns. section 10.1.4 section 10.3.2
Use deletion vectors within a merge-on-read design, while accounting for these tradeoffs:
- Record sparse updates or deletes without rewriting entire data files, improving write throughput while deferring reorganization. section 10.3.2
- Apply deletion information during reads to reconstruct the current table view; read complexity is lower than with accumulating delete files but still requires efficient maintenance. section 10.1.4
- Treat logical deletion separately from physical deletion: old snapshots can preserve deleted rows until compaction, snapshot expiration, and orphan cleanup remove unreferenced files. section 11.2.5
Sparse correction or compliance delete
For a GDPR request or a correction affecting a small population of sensors or user profiles, record the affected rows in deletion vectors instead of rewriting complete data files. Later maintenance can compact files and enforce physical removal. section 10.3.2 section 11.2.5
Figure 10.6 In Copy-on-Write, when you delete records, the data file with those records is rewritten without them. In Merge-on-Read, instead of rewriting a whole file, the deleted records are tracked in a delete file or deletion vector. This can be used to reconcile the table when it is read, which enables faster writing to the table but slows down reading.
Compare copy-on-write file rewrites with merge-on-read deletion tracking and its write-versus-read performance tradeoff. section 10.1.4
