A self-healing mechanism detects when an actual system state differs from its desired state and automatically restores correct operation. It turns resilience from failure containment into active correction, reducing operational friction for teams building, releasing, and operating software. section 7.0
Implement self-healing most directly where healthy states and failure conditions are known:
- Define telemetry, queries, and alerts for violations such as sustained Kubernetes CPU utilization above 90%, then trigger corrective action. section 5.1.3
- Use Kubernetes orchestration to redeploy failing services, move workloads from failing virtual machines, or adjust instance counts with traffic. section 7.0
- Extend the pattern through the Kubernetes API when platform components need correction beyond standard service orchestration. section 7.0
- Instrument unknown failure paths with telemetry and tracing; self-healing is easiest where the violated healthy state is already understood. section 5.1.3
Project-creation service
Suppose load testing establishes that an API creating source-control projects handles six simultaneous requests, while production traffic reaches more. Define the healthy capacity, observe the violation, and use platform orchestration to restore service capacity quickly. section 5.1.3
Figure 5.3 Knowing what to look for when querying observability data is an evolving process.
Use the observability progression to distinguish failures that support explicit self-healing from problems that first require better telemetry and diagnosis. section 5.1.3
