Hard delete physically removes deleted data from the files or objects that store it. In Apache Iceberg, deleting a row first changes table state; physical removal waits until snapshots no longer reference the relevant files and cleanup completes. This separation supports snapshot isolation, time travel, performance, and auditability, but complicates regulations requiring timely, irrevocable erasure. section 10.3.3 section 11.2.5
Enforce hard deletion through a coordinated lifecycle:
- Expire every snapshot that can reference records or files containing the deleted data. section 10.3.3
- Compact Merge-on-Read tables so delete files or deletion vectors are applied to base files and deleted rows are excluded. section 11.2.5
- Confirm that rewritten files are unreferenced by retained snapshots before removing their predecessors. section 10.3.3
- Run orphan-file cleanup and monitor object storage, because metadata cannot prove that physical removal succeeded. section 11.2
Copy-on-Write rewrites affected files during the delete, while Merge-on-Read records separate delete files and leaves original data files intact until compaction. In either mode, older snapshots can preserve access to logically deleted rows until explicitly expired. section 11.2.5
Auditing candidate deleted files
A metadata query can identify files associated with DELETE operations in the EU partition that have not changed for at least 30 days. Treat this as traceability, not proof of storage deletion; pair it with snapshot expiration, orphan cleanup, and storage monitoring. section 11.2
SELECT f.file_path FROM my_table.files f LEFT JOIN my_table.metadata_log_entries m ON f.file_path = m.file WHERE m.operation = 'DELETE' AND f.partition = 'region=EU' AND f.last_modified <DATE_SUB(CURRENT_DATE, INTERVAL 30 DAY);
Inspect file metadata and metadata-log entries to trace deleted files, partitions, operations, and modification age. section 11.2
