liveBook · Manning

Snapshot lineage

As explained in Architecting an Apache Iceberg Lakehouse

Snapshot lineage is the history of relationships and changes among versioned table snapshots over time.

Snapshot lineage records how versioned Iceberg table states relate and change over time. Each write creates a snapshot that points to its parent, forming a directed lineage rather than a strictly chronological log. section 11.2.1

Use lineage metadata to:

  • Inspect snapshot IDs, parent IDs, operations, and summaries of added, removed, or rewritten records. section 11.2.1
  • Trace an unexpected row-count change from a commit to the specific files added or removed through manifest metadata. section 11.2.1
  • Support auditing, governance, rollback, historical queries, and recovery alongside retention and storage controls. section 11.0 section 10.4

Investigating a changed fact table

Query the table’s snapshots metadata table, order recent commits for inspection, compare summary metrics, and drill into manifests to locate the responsible files. Treat committed timestamps as informative rather than perfectly reliable ordering signals. section 11.2.1

SELECT snapshot_id, committed_at, operation, summary
FROM "db "."critical_fact_table.snapshots "ORDER BY committed_at DESC;

Use the supplied SQL listing to inspect snapshot IDs, commit timestamps, operations, and summary statistics for a table. section 11.2.1

How Architecting an Apache Iceberg Lakehouse covers it

How deep it goes
Explains parent-child lineage, snapshot summaries, timestamp limitations, and manifest-level investigation, but not full recovery implementation.
Examples
Uses a critical fact-table row-count investigation with snapshot SQL, summary metrics, and manifest inspection.
What you can do after
Diagnose anomalous table changes by identifying commits, comparing record metrics, and tracing affected files.
How it connects
Connects versioning, time travel, metadata tables, manifests, auditing, rollback, and disaster recovery.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Architecting an Apache Iceberg Lakehouse. The section links point to the parts of the book it draws on.