liveBook · Manning

Permanent failure

As explained in Think Distributed Systems

A failure whose subsequent occurrence is certain and whose resolution requires manual intervention.

A permanent failure is a failure whose recurrence is certain after it occurs and whose resolution requires manual intervention. Unlike failures that repair themselves, it does not resolve automatically. section 3.3.3

Classify failures by what happens after the first occurrence:

  • Transient failures do not increase the probability of another failure and repair automatically. section 3.3.3
  • Intermittent failures increase the probability of another failure but still repair automatically. section 3.3.3
  • Permanent failures make another occurrence certain and require manual repair. section 3.3.3

An employee leaves permanently

In a distributed-company analogy, an employee taking a short break models a transient failure, a vacation models an intermittent failure, and leaving the company models a permanent failure. The departure does not resolve by waiting; the organization must intervene. section 1.6

Figure 3.14 Permanent failure
A math equation with a line

AI-generated content may be incorrect.

Use the failure-classification figure to compare permanent failure with transient and intermittent failures along the temporal dimension. section 3.3.3

How Think Distributed Systems covers it

How deep it goes
Defines permanent failure through conditional probability and manual repair, contrasting it with two autorepairing failure types.
Examples
Uses a departing employee to model permanent failure within a distributed-company analogy.
What it assumes
Assumes a basic distributed-systems setting but builds the failure distinction from everyday absences and departures.
What you can do after
Evaluate whether a failure is self-repairing or requires intervention by examining recurrence likelihood and repair behavior.
How it connects
Connects permanent failure with transient failures, intermittent failures, crash semantics, and message-delivery semantics.

Also explained in

Compare how 2 books explain permanent failure →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Think Distributed Systems. The section links point to the parts of the book it draws on.