liveBook · Manning

Resilience

As explained in Think Distributed Systems

The ability of a system or protocol to continue providing acceptable behavior despite component or network failures.

Fault tolerance is the ability of a system to behave in a well-defined manner despite failures. A failure moves the system from a legal state toward an illegal state; intolerable states are beyond the guarantees being considered. section 3.0 section 3.1

A useful mental model treats the system as states connected by normal and failure transitions. Recovery is the sequence of transitions that returns an illegal state to a legal state. section 3.1

Classify tolerance by the guarantees preserved during failure:

  • Masking tolerance preserves both safety and liveness. section 3.2
  • Nonmasking tolerance preserves liveness while temporarily compromising safety. section 3.2
  • Fail-safe tolerance preserves safety while giving up liveness. section 3.2
  • Preserving neither property is not fault tolerance. section 3.2.4

A single-variable system

Let process P contain x ∈ {0, 1, 2, 3}. Define safety as x being 1 or 2, and liveness as eventually reaching both values. Normal transitions alternate 1 and 2; repair moves 0 to 1; a failure moves any nonzero value to 0. The system is nonmasking: x = 0 violates safety but can recover and remain live. State 3 is intolerable because no return guarantee exists. section 3.2.4

How Think Distributed Systems covers it

How deep it goes
Defines states, transitions, recovery, safety, and liveness, then stops short of detailed implementation mechanisms.
Examples
Uses a one-process variable with legal, illegal, and intolerable values plus normal, repair, and failure transitions.
What it assumes
Assumes system models, distributed execution, ordering, and logical or physical time as foundational concepts.
What you can do after
Evaluate failure behavior by identifying legal states, failure transitions, recovery paths, and preserved safety or liveness guarantees.
How it connects
Connects distributed systems with system models, failure recovery, safety, liveness, and failure handling.

Also explained in

Compare how 13 books explain resilience →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Think Distributed Systems. The section links point to the parts of the book it draws on.