liveBook · Manning

Data loss tolerance

As explained in Kafka for Architects

Data loss tolerance is the maximum amount of expected or acceptable data loss under failure conditions.

Data loss tolerance is a nonfunctional requirement that defines how much message loss is acceptable during broker outages, network partitions, or system crashes. It shapes persistence, acknowledgment, fallback, and operational decisions. section 10.1.4

Classify tolerance according to business impact:

  • Critical: accept no message loss; persist events and fully acknowledge them. section 10.1.4
  • Important: tolerate occasional loss only when fallback mechanisms exist. section 10.1.4
  • Non-critical: use best-effort delivery when loss has minimal business impact. section 10.1.4

Failure-policy decision

A message stream exposed to broker outages should first be classified as critical, important, or non-critical. That classification determines whether the design requires full acknowledgment, fallback handling, or best-effort delivery. section 10.1.4

How Kafka for Architects covers it

How deep it goes
Defines three tolerance classes and links them to persistence, acknowledgment, and fallback, without detailing implementation.
Examples
Uses broker outages, network partitions, system crashes, and critical, important, and non-critical delivery classifications.
What you can do after
Evaluate delivery requirements and choose persistence, acknowledgment, or fallback expectations for failure conditions.
How it connects
Connects data integrity, availability, persistence, acknowledgments, fallback mechanisms, and dead-letter handling.
Cover of Kafka for Architects
liveBookmanning.com
Read Kafka for Architects on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.