liveBook · Manning

Deletion semantics

As explained in Architecting an Apache Iceberg Lakehouse

Deletion semantics are the rules determining how a deletion changes logical table state and when it removes corresponding physical data.

Deletion semantics distinguish a logical table change from physical removal. A delete updates the current snapshot, but deleted rows can remain in data files and older snapshots until cleanup completes. section 11.2.5

The deletion path differs by table layout:

  • Copy-on-Write rewrites affected data files immediately, producing versions without deleted or updated rows. section 11.2.5
  • Merge-on-Read records position deletes or deletion vectors separately while retaining the original data file for read-time filtering. section 11.2.5
  • Hard deletion requires coordinating compaction, snapshot expiration, and orphan cleanup so unreferenced files can be removed. section 11.2.5
Figure 11.3 When a record is deleted, a new snapshot is created that doesn’t include the deleted data, but the data files still exist. Only once all snapshots associated with the files containing the deleted data have expired are those data files actually removed from storage.

Trace how a new snapshot hides deleted data before snapshot expiration finally permits physical file removal. section 11.2.5

How Architecting an Apache Iceberg Lakehouse covers it

How deep it goes
Explains COW and MOR mechanics and cleanup sequencing, but does not detail specific operational commands.
Examples
Uses data files, delete files, deletion vectors, snapshots, and a deleted record to illustrate logical and physical removal.
What it assumes
Assumes familiarity with table snapshots and data files while distinguishing COW and MOR deletion behavior.
What you can do after
Operate/configure deletion workflows by coordinating compaction, snapshot expiration, and orphan cleanup.
How it connects
Connects deletion semantics with snapshots, time travel, compaction, orphan cleanup, and Copy-on-Write and Merge-on-Read models.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Architecting an Apache Iceberg Lakehouse. The section links point to the parts of the book it draws on.