Delivery guarantee

As explained in Kafka for Architects

A delivery guarantee is a defined assurance about whether and how messages are delivered, retained, and processed despite failures or retries.

A delivery guarantee defines whether messages are sent, retained, acknowledged, and processed despite failures, retries, lost acknowledgments, and consumer restarts. In asynchronous systems, proving where a message disappeared is difficult, so guarantees depend on explicit producer, broker, and consumer behavior. section 7.4 section 5.1.1

Common failure paths include:

• A producer never sends the message, or fails to handle a missing acknowledgment after the broker rejects or loses the write. section 7.4
• A leader acknowledges a message before failing to replicate it to followers, creating potential data loss. section 7.4
• Retention can delete a message before consumption, while a consumer can reread processed data after failing before committing its offset. section 7.4
• A lost acknowledgment can cause a producer to resend an already written batch, creating duplicates; retries can also raise ordering concerns. section 7.4 section 3.2.5

Customer 360 producer configuration

A producer can strengthen delivery behavior with enable.idempotence=true, acks=all, appropriate retries, and a suitable max.inflight.requests.per.connection value. The same architecture uses entity identifiers as partitioning keys and unique client identifiers for monitoring. These settings support ordered, duplicate-free batch delivery under the stated criteria, while acks=all provides assurance against broker-side data loss. section 3.2.5

Figure 5.2 Using compacted topics in Kafka: you always retain the latest version of each event, allowing Kafka to act as a source of truth.

Compacted topics illustrate the retention dimension of delivery guarantees by preserving the latest version of each event and allowing Kafka to act as a source of truth. section 5.1.1

How Kafka for Architects covers it

How deep it goes

Explains producer, broker, retention, and consumer failure modes, then connects them to configuration-based guarantees.

Examples

Uses lost acknowledgments, replicated writes, consumer retries, Customer 360 producer settings, and compacted topics as concrete examples.

What it assumes

Assumes familiarity with producers, consumers, acknowledgments, partitions, retries, offsets, and retention policies.

What you can do after

Evaluate Kafka producer configurations and failure paths for potential loss, duplication, reprocessing, and ordering effects.

How it connects

Connects Kafka delivery guarantees with producer idempotence, acknowledgments, retries, partition ordering, retention policies, and consumer offsets.

liveBookmanning.com

Read Kafka for Architects on liveBook Get the book at manning.com

Related concepts

• redundancyaddresses · Redundancy is the presence of duplicate or overlapping information, components, or resources, whether intentional or unnecessary.
• data lossaddresses · Data loss is the destruction, corruption, deletion, or unavailability of information that is expected to be retained or remain usable.
• acknowledgment strategyconfigured by · An acknowledgment strategy is a producer-side policy determining when a published message is acknowledged and what delivery guarantee applies.
• at-most-once deliveryincludes · At-most-once delivery is a messaging guarantee that each message is delivered no more than once, so messages may be lost but are not duplicated.

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.