A delete file is an auxiliary Iceberg data structure that records rows to exclude from query results without immediately rewriting the underlying data files. It supports merge-on-read updates and deletes, reducing write amplification while moving reconciliation work into reads. section 6.3.1 section 10.1.4
Iceberg uses two principal delete-file forms:
- Position deletes identify rows by their data file and physical row position; they are generally faster to apply during reads and easier to compact. section 6.3.1
- Equality deletes identify rows through column-value predicates; they suit streaming and change-data-capture pipelines but cost more to evaluate and compact. section 6.3.1 section 10.1.4
- Copy-on-write avoids delete files by rewriting affected data files, trading lower read complexity for greater write amplification and update latency. section 6.3.1
Figure 6.9 When using delete files, updated and deleted records in existing data files are ignored, while new data files containing only the updated records are written.
Shows how delete files cause updated or deleted rows in existing data files to be ignored while replacement records are written separately. section 6.3.1
Each merge-on-read query scans base data files and applies relevant delete files to reconstruct the current table view. As delete files accumulate, especially equality deletes, query complexity, read latency, and storage costs rise. Maintenance can monitor file counts and sizes, then rewrite position-delete files or compact affected data. section 10.1.4 section 11.1.2
A logical deletion does not immediately remove the original row from object storage. Older snapshots can still expose it through time travel until snapshots expire and orphaned or unreferenced files are cleaned up; hard deletion may therefore require compaction followed by coordinated snapshot and file cleanup. section 11.2.5
