Secure data deletion in an Iceberg lakehouse requires removing both the logical record and every retained physical representation. A delete changes the table snapshot first, but older snapshots, data files, and delete files can preserve the data on object storage. Compliance therefore depends on coordinating table semantics with snapshot expiration, compaction, and orphan-file cleanup. section 11.2.5
Deletion behavior differs by table layout:
- Copy-on-Write rewrites affected data files immediately, creating versions without the deleted rows; obsolete files remain until no retained snapshot references them. section 11.2.5
- Merge-on-Read records position deletes or deletion vectors separately, leaving original data files intact until compaction rewrites them without the deleted rows. section 11.2.5
- A hard-deletion workflow expires snapshots that can expose the data, compacts MOR delete files when needed, and removes unreferenced files through orphan cleanup. section 11.2.5 section 11.4
Figure 11.3 When a record is deleted, a new snapshot is created that doesn’t include the deleted data, but the data files still exist. Only once all snapshots associated with the files containing the deleted data have expired are those data files actually removed from storage.
Trace how a deleted record remains in physical files after a new snapshot omits it, then becomes removable after all referencing snapshots expire. section 11.2.5
