Fault tolerance keeps a system providing acceptable service when bugs, component failures, overload, network problems, or machine crashes occur. It localizes failures, minimizes their effects, detects them, and recovers automatically instead of allowing one error to take down the entire system. section 8.0 section 1.1.1
BEAM systems pursue fault tolerance through several cooperating mechanisms:
- Isolate work in independent BEAM processes that share no memory, so one crash does not directly corrupt or stop unrelated work. section 1.1.2 section 8.2
- Detect failed processes and recover by restarting them or reconnecting dependent components. section 1.1.2 section 8.0
- Use process links and supervisors to propagate failures where necessary and coordinate self-healing without burdening the main code with defensive error handling. section 12.2.6
- Keep partial service available during dependency failures, such as serving cached data and queueing writes while a database connection is restored. section 8.0
- Distribute nodes across machines so another machine can continue service when one crashes, while preparing for broken connections and netsplits. section 12.4
An isolated process failure
One spawned process raises an exception while another sleeps and later prints that it finished. The failed process reports an error, but the second process and the interactive shell continue running. A related to-do architecture isolates individual to-do servers from one another and lets cached reads continue when the database process crashes. section 8.2 section 8.2
Shows how the BEAM schedules many isolated processes within a single runtime, the foundation for fine-grained fault containment. section 1.1.2
