Logical deletion changes a table’s logical state so queries exclude records or files while their physical data remains in storage. In Apache Iceberg, delete operations create metadata changes and new snapshots rather than immediately removing files. This preserves snapshot isolation and time travel, but delays irrevocable erasure. section 10.3.3 section 11.2.5
Treat logical deletion as the first stage of a controlled physical-erasure process:
- Expire snapshots that reference the affected records or files; retained snapshots keep those files accessible. section 10.3.3
- Compact merge-on-read tables or rewrite data files so deleted rows are physically excluded; deletion vectors reduce fragmentation but do not eliminate the need for compaction. section 10.3.3 section 11.2.5
- Run orphan-file cleanup only after expiration and compaction, then monitor storage to verify that unreferenced files were actually removed. section 10.3.3 section 11.2
SELECT f.file_path FROM my_table.files f LEFT JOIN my_table.metadata_log_entries m ON f.file_path = m.file WHERE m.operation = 'DELETE' AND f.partition = 'region=EU' AND f.last_modified <DATE_SUB(CURRENT_DATE, INTERVAL 30 DAY);
Inspect file metadata and deletion entries to identify logically deleted files in a regional partition that have exceeded a retention threshold. section 11.2
Auditing a deletion
A defensible audit combines metadata evidence—such as delete or rewrite entries and the absence of files from active snapshots—with snapshot expiration, orphan cleanup, and storage-level monitoring. Metadata proves the table’s logical state, not successful physical removal. section 11.2
