liveBook · Manning

Change failure rate

As explained in Infrastructure as Code, Patterns and Practices: With examples in Python and Terraform

Change failure rate is the proportion of deployments or changes that require remediation, rollback, or other corrective action because they fail.

Change failure rate measures the proportion of deployments or infrastructure changes that fail and require remediation, rollback, or other corrective action. Manual configuration tends to increase it because changes accumulate without consistent auditing or organization. section 1.2.1

A manual infrastructure failure pattern

A team waited nearly two months for a database environment while platform staff handled database creation, routing, and firewall tickets. Later changes were harder to troubleshoot because earlier modifications were not organized or audited. section 1.2.1

Reduce change failure rate by:

  • Writing infrastructure as code so systems can be reproduced and changes can be reviewed. section 1.4.2
  • Using automation and tests to improve predictability and limit the blast radius of failed changes. section 1.4.2
  • Investing upfront in IaC to reduce debugging and recovery effort as infrastructure evolves and scales. section 1.4.2

How Infrastructure as Code, Patterns and Practices: With examples in Python and Terraform covers it

How deep it goes
Defines the metric and explains causes and mitigations, but does not provide a calculation method or reporting procedure.
Examples
Uses delayed database provisioning, routing and firewall tickets, and unaudited firewall changes as operational examples.
What you can do after
Apply it via infrastructure as code, automation, and testing to reduce failed changes and remediation effort.
How it connects
Links change failure rate to infrastructure as code, automation, testing, predictability, blast radius, and MTTR.

Also explained in

Compare how 3 books explain change failure rate →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Infrastructure as Code, Patterns and Practices: With examples in Python and Terraform. The section links point to the parts of the book it draws on.