Fault tolerance
Fault tolerance is the ability of a system to continue providing correct or acceptable service despite failures in its components or operating environment.
ScyllaDB in Action
Fault tolerance is the ability of a system to continue providing correct or available service despite component failures.
- How it is used
- Frames replicated, distributed storage as the way ScyllaDB remains available when nodes fail.
- How deep it goes
- Explains replication, peer routing, eventual consistency, and acknowledgment thresholds, then verifies quorum behavior by stopping a node.
- Examples
- Uses a three-node cluster with replication factor three, QUORUM queries, and one stopped node.
- What you can do after
- Operate/configure it by choosing replica acknowledgments that keep queries available through planned node failures.
Akka in Action, Second Edition
Fault tolerance is the ability of a system to continue operating and preserve important services when components or operations fail.
- How it is used
- Frames reliable concurrent and distributed applications as recovery design rather than impossible fault prevention.
- How deep it goes
- Explains containment, replacement, reboot, actor supervision, and a clustered node-failure scenario; defers broader catastrophic protection to clustering.
- Examples
- Demonstrates a text-processing cluster whose master accumulates lag while aggregators fail, then resumes delegation after a replacement joins.
- What you can do after
- Apply it via Akka supervision and clustering to keep message-based work progressing through actor or node failures.
Read how Akka in Action, Second Edition explains fault tolerance →
Bootstrapping Microservices, Second Edition: With Docker, Kubernetes, GitHub Actions, and Terraform
Fault tolerance is the ability of a system to continue providing acceptable service when some components fail.
- How it is used
- Builds reliable microservices that continue serving customers during component failures and recover from errors.
- How deep it goes
- Explains replicas, health checks, timeouts, retries, queues, and circuit breakers, with JavaScript and Axios examples but limited internals.
- Examples
- Uses Kubernetes replicas, Axios timeouts and retries, internal services, and a Dropbox integration as concrete failure scenarios.
- What you can do after
- Build it from scratch using replicas, health checks, timeouts, retries, queues, and circuit breakers.
Apache Kafka in Action
Fault tolerance is the ability of a system to continue operating correctly despite failures in components or infrastructure.
- How it is used
- Keeps Kafka event flows recoverable through component failures and guides Kafka Connect worker and error-handling choices.
- How deep it goes
- Explains task reassignment and specific error settings, but does not detail Kafka’s underlying storage failure mechanisms.
- Examples
- Uses a 12-table database import, a worker-crash scenario, faulty connector records, and a retailer splitting workloads across clusters.
- What you can do after
- Operate/configure it by distributing Connect tasks and choosing logging, skipping, dead-letter queues, or retries for connector failures.
Designing Deep Learning Systems: A guide for software engineers
The ability of a system to continue operating correctly or recover when components or operations fail.
- How it is used
- Protects distributed training and hyperparameter-optimization workflows from losing all progress after worker, node, or network failures.
- How deep it goes
- Explains checkpoint storage, worker restart, and state restoration, but does not detail checkpoint formats or recovery orchestration.
- Examples
- Uses remote model-state storage, TensorFlow and PyTorch backup features, and failed sequential HPO trials caused by node or network problems.
- What you can do after
- Operate/configure training services to store checkpoints remotely and resume failed workers or sequential optimization trials.
Read how Designing Deep Learning Systems: A guide for software engineers explains fault tolerance →
Related concepts
- transaction logaddressed by · A transaction log is an ordered record of transactions, transaction-related events, or resulting data changes, maintained as a history and often used for tracking or recovery.
- resiliencecontrasts with · Resilience is the capacity of a system or organization to withstand disruption, adapt, recover, or continue functioning acceptably.
- stretched clusterproduced by · A stretched cluster is a distributed cluster whose nodes are deployed across multiple physical locations or failure domains.
- failureaddresses · A failure is an unsuccessful outcome or condition in which an operation, process, or system does not achieve an expected result.
- human-in-the-loopimplemented by · Human-in-the-loop is a system pattern in which people provide input, judgment, review, or intervention within an automated process.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.