dbt provides a lightweight, model-driven way to keep Iceberg maintenance close to transformation logic. A model can process new data, then a run-operation macro can invoke SQL procedures such as snapshot expiration. section 11.1.1 section 11.1.5
A dbt-based Iceberg workflow can:
- Run a transformation model against source data. section 11.1.5
- Invoke parameterized maintenance macros with dbt run-operation. section 11.1.1 section 11.1.5
- Chain cleanup after ingestion while accepting less scheduling and DAG control than Airflow. section 11.1.1
-- models/transform_customer_data.sql
SELECT *
FROM raw.customers
WHERE updated_at >CURRENT_DATE - INTERVAL '30 days'
-- macros/expire_snapshots.sql
{% macro expire_snapshots(table_name) %}
CALL {{ table_name }}.system.expire_snapshots(
older_than =>CURRENT_TIMESTAMP - INTERVAL '7 days'
);
{% endmacro %}
Inspect a dbt model and macro that filter recent customer data and expire old Iceberg snapshots. section 11.1.5
