Data loss
Data loss is the destruction, corruption, deletion, or unavailability of information that is expected to be retained or remain usable.
Kafka for Architects
Data loss is the permanent or temporary unavailability or destruction of data that a system was expected to preserve.
- How it is used
- Frames data loss as an end-to-end delivery risk and a nonfunctional tolerance requirement for Kafka systems.
- How deep it goes
- Explains concrete failure paths and offset and dead-letter handling, but does not specify producer acknowledgment or replication configurations.
- Examples
- Illustrates loss with a leader failing before replication, a committed offset preceding failed processing, and schema-incompatible messages diverted from a sink.
- What you can do after
- Diagnose/debug with it by tracing message stages and inspecting offset commitments and dead-letter topics when events go missing.
Apache Kafka in Action
Data loss is the unintended destruction or unavailability of stored data.
- How it is used
- Protects Kafka data by linking producer acknowledgments, replica availability, disk capacity, and recovery operations.
- How deep it goes
- Explains failure mechanisms and operational mitigations, while deferring detailed producer-send behavior.
- Examples
- Uses stopped brokers, topic descriptions, cola messages, full volumes, failed disks, and NOT_ENOUGH_REPLICAS errors.
- What you can do after
- Operate/configure Kafka with replication and storage practices that reduce accepted writes and data loss during failures.
Architecting an Apache Iceberg Lakehouse
Data loss is the destruction or unavailability of data that prevents its intended recovery or use.
- How it is used
- Protects Apache Iceberg tables against storage failures, accidental deletion, corruption, and malicious actions.
- How deep it goes
- Explains versioning, permissions, recovery scenarios, and validation checks, but stops short of complete restoration procedures.
- Examples
- Uses S3 Versioning, Azure Blob Snapshots, immutable storage, archived snapshots, and read-only scan jobs as safeguards or tests.
- What you can do after
- Operate/configure recovery safeguards and validate whether Iceberg snapshots remain usable after storage or metadata failures.
Read how Architecting an Apache Iceberg Lakehouse explains data loss →
API Design Patterns
Data loss is the unintended removal or destruction of information that should have been retained.
- How it is used
- Protects API resources from accidental overwrites during partial changes, concurrent updates, version evolution, and deletion.
- How deep it goes
- Explains concurrent replacement and stale schemas through a worked ChatRoom sequence, then contrasts soft deletion with backups.
- Examples
- Uses concurrent ChatRoom clients and contrasts permanent deletion with backup or soft-deletion policies.
- What you can do after
- Diagnose/debug data loss by identifying stale replacements, missing consistency checks, outdated fields, or unsuitable deletion policies.
Related concepts
- backup and restoreaddressed by · Backup and restore is the practice of preserving copies of data or system state and using them to recover it after loss, damage, or failure.
- quorumaddressed by · A quorum is the minimum number or proportion of distributed participants whose responses or agreement are sufficient for an operation or decision.
- recovery point objectivemeasured by · A recovery point objective is the maximum acceptable amount of data loss, typically expressed as the time between a disruption and the latest recoverable state.
- unclean leader electionproduced by · Unclean leader election is the promotion of a non-in-sync replica to partition leader when no eligible in-sync replica is available.
- data corruptionproduced by · Data corruption is the unintended alteration, loss, or damage of data that makes it inaccurate, invalid, inconsistent, or unusable.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.