An observability tool collects, stores, and visualizes operational signals so teams can monitor system health, diagnose problems, and respond to anomalies. In Iceberg operations, metadata-derived metrics expose file counts, snapshot sizes, query latency, compaction frequency, and catalog access patterns. section 11.1.4
Common deployment choices reflect different operational tradeoffs:
- Prometheus provides pull-based time-series monitoring and commonly supports Kubernetes or containerized environments. section 11.1.4
- CloudWatch aggregates AWS metrics, logs, and events, making it a natural fit for Iceberg deployments using S3 and Glue. section 11.1.4
- Datadog combines logs, metrics, traces, dashboards, alerting, and correlation through a commercial SaaS platform. section 11.1.4
Compaction failure alert
An orchestration log records a failed rewrite_data_files operation caused by too many open files, then triggers a maintenance.compaction_failure alert. section 11.1.4
[2025-10-01 02:00:12,345] {taskinstance.py:1064} INFO - Starting task: compact_orders_table
[2025-10-01 02:00:13,678] {sql.py:76} INFO - Running SQL: CALL iceberg.system.rewrite_data_files('db.orders')
[2025-10-01 02:00:20,982] {sql.py:81} ERROR - Compaction failed: Too many open files
[2025-10-01 02:00:21,104] {taskinstance.py:1234} ERROR - Task failed with exception: OSError: [Errno 24] Too many open files
[2025-10-01 02:00:21,105] {taskinstance.py:1300} INFO - Marking task as FAILED
[2025-10-01 02:00:21,200] {taskinstance.py:1421} INFO - Triggering alert: maintenance.compaction_failure
Inspect a realistic maintenance log showing task execution, the underlying error, failure state, and alert trigger. section 11.1.4
