Fault tolerance

As explained in Apache Kafka in Action

Fault tolerance is the ability of a system to continue operating correctly despite failures in components or infrastructure.

An order-processing system cannot stop whenever a downstream service fails. Kafka’s persistent, distributed event log lets a consumer resume where it left off and replay stored events. Its distributed design also lets a cluster continue handling data when parts of the system fail. section 1.1

For Kafka Connect, fault tolerance depends on where tasks can run. Workers in Distributed Mode share a group.id and coordinate task assignments. If a worker crashes, the remaining workers redistribute its tasks; adding a worker redistributes work for scaling. A connector that reads only a worker’s local files is a poor fit for this approach, so the FileStreamSource connector is not recommended for production. section 11.3

Distribute a database import

To import 12 database tables, set tasks.max to 4: each task can handle three tables, and Connect distributes the tasks across workers. The tasks provide parallel work; multiple workers give Connect places to reassign that work after a worker failure. section 11.3

Figure 11.4 A Kafka Connect cluster consists of one or multiple workers. Tasks are distributed across workers, and Kafka Connect uses three topics to store its internal state.

The cluster diagram shows tasks spread across workers and three topics holding Connect’s internal state. section 11.3

A running worker does not mean every record can be processed. Choose an error policy for the data and external systems involved:

• By default, a connector fails on an error. Setting errors.tolerance to all skips faulty records instead; enable errors.log.enable and consider alerts so skipped work remains visible. section 11.6.2
• For a sink connector, errors.deadletterqueue.topic.name can direct faulty messages to a dead-letter queue. Enable errors.log.include.messages only after considering whether records contain sensitive data. section 11.6.2
• For temporary access problems, configure errors.retry.timeout and errors.retry.delay.max.ms. The retry timeout defaults to zero, so retries must be enabled deliberately. section 11.6.2

At enterprise scale, fault tolerance is also a workload-planning concern. A global retailer might separate transactional, inventory, and analytics workloads into Kafka clusters so each can grow without overloading one shared cluster. This is a reason to split workloads, not a substitute for configuring worker recovery and record-error handling. section 18.4.4 section 11.6.2

How Apache Kafka in Action covers it

How deep it goes

Explains task reassignment and specific error settings, but does not detail Kafka’s underlying storage failure mechanisms.

Examples

Uses a 12-table database import, a worker-crash scenario, faulty connector records, and a retailer splitting workloads across clusters.

What it assumes

Introduces producers, consumers, topics, and distributed workers in context rather than assuming prior Kafka knowledge.

What you can do after

Operate/configure it by distributing Connect tasks and choosing logging, skipping, dead-letter queues, or retries for connector failures.

How it connects

Connects fault tolerance to persistent logs, consumer replay, Kafka Connect distributed workers, dead-letter queues, retries, and cluster scaling.

liveBookmanning.com

Read Apache Kafka in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  ScyllaDB in ActionFault tolerance is the ability of a system to continue providing correct or available service despite component failures.
• 

  liveBookmanning.com
  Akka in Action, Second EditionFault tolerance is the ability of a system to continue operating and preserve important services when components or operations fail.
• 

  liveBookmanning.com
  Bootstrapping Microservices, Second Edition: With Docker, Kubernetes, GitHub Actions, and TerraformFault tolerance is the ability of a system to continue providing acceptable service when some components fail.
• 

  liveBookmanning.com
  Designing Deep Learning Systems: A guide for software engineersThe ability of a system to continue operating correctly or recover when components or operations fail.

Compare how 5 books explain fault tolerance →

Related concepts

• transaction logaddressed by · A transaction log is an ordered record of transactions, transaction-related events, or resulting data changes, maintained as a history and often used for tracking or recovery.
• resiliencecontrasts with · Resilience is the capacity of a system or organization to withstand disruption, adapt, recover, or continue functioning acceptably.
• stretched clusterproduced by · A stretched cluster is a distributed cluster whose nodes are deployed across multiple physical locations or failure domains.
• failureaddresses · A failure is an unsuccessful outcome or condition in which an operation, process, or system does not achieve an expected result.
• human-in-the-loopimplemented by · Human-in-the-loop is a system pattern in which people provide input, judgment, review, or intervention within an automated process.

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.