liveBook · Manning

Durable execution

As explained in Think Distributed Systems

Durable execution is an execution model that persists workflow progress and state so execution can resume after failures without requiring application logic to handle those failures explicitly.

Durable execution persists workflow progress and state so an execution can resume after failure without requiring application logic to detect and mitigate failures explicitly. It provides failure transparency: a failure-free run and a failed-then-recovered run appear equivalent. section 11.0 section 11.6

Two implementation strategies preserve progress differently:

  • Log-based execution records each step’s output, replays the process after failure, and reuses previously recorded results. It requires deterministic execution and can accumulate a large log. section 11.5.2
  • State-based execution records the execution state, or continuation, after each step; recovery restores that continuation and resumes without replaying completed steps. section 11.5.2

Creating an account and collecting payment

A signup workflow yields account creation and payment processing as separate steps. If execution fails between them, durable recovery preserves the completed work and continues from the appropriate point, while the signup definition remains focused on business logic. section 11.5.1 section 11.5.2

Listing 11.8 Durable execution: State-based implementation
async function runtime(coroutine, args, durableState) {
  let execution;

  // If a durable state exists, load and resume the coroutine,
➥otherwise start a new one.
  if (durableState) {
    execution = load(durableState);
  } else {
    execution = coroutine(args);
  }

  // Execute the coroutine steps.
  let stepResult = execution.next();

  while (!stepResult.done) {
    // Save the current state.
    durableState = save(execution);

    // Execute the current step and get the result.
    const value = await stepResult.value();

    // Move to the next step.
    stepResult = execution.next(value);

  }
}

Inspect a compact state-based runtime that loads saved execution state, saves the continuation, executes a step, and resumes with its result. section 11.5.2

How Think Distributed Systems covers it

How deep it goes
Explains failure transparency and two recovery strategies, including replay, continuations, determinism, and log growth.
Examples
Uses a signup workflow with account creation, payment processing, coroutines, durable logs, and saved execution state.
What you can do after
Apply it via log-based or state-based runtimes to resume multi-step workflows after execution failures.
How it connects
Connects distributed systems, transactions, failure transparency, sagas, deterministic execution, and continuations.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Think Distributed Systems. The section links point to the parts of the book it draws on.