Fail-fast ingestion rejects records or halts processing when incoming data violates the expected schema. It protects Iceberg tables from invalid data and exposes schema problems early, but can cause repeated failures and downtime when source quality varies. section 6.1.3
Apply fail-fast checks to:
- Data type mismatches that cannot safely conform to the target definition. section 6.1.3
- Missing fields or other schema inconsistencies in incoming records. section 6.1.3
- Critical fields where preventing invalid records matters more than uninterrupted processing. section 6.1.3
Selective enforcement
A pipeline validates incoming records against the current Iceberg table definition and stops ingestion when a critical field has an incompatible type, rather than allowing the record into the table. section 6.1.3
