On-call duty is the operational responsibility of responding to incidents and urgent service problems during designated coverage periods. Effective support combines timely response with practices that restore stability and prevent repeat failures. section 12.2.6 section 10.1.5
A practical on-call arrangement includes:
- Monitoring and alerts that notify the on-call engineer when something goes wrong. section 10.1.5
- Service-level agreements that define expected response or resolution times according to impact. section 10.1.5
- Runbooks and troubleshooting documents that record common problems and resolutions. section 10.1.5
- Handoff meetings that transfer recent incidents, key information, and lessons between outgoing and incoming engineers. section 12.2.6
- Root-cause work that prevents recurring failures instead of treating every alert as an isolated event. section 12.2.6
Example: matching response to severity
A business-wide outage or customer-data security breach is treated as the highest severity: work stops and the problem receives immediate attention, with a four-to-six-hour SLA. A smaller checkout problem affecting discount-code users can receive a lower severity and a 24-hour SLA. section 10.1.5
Table 10.2 Different severity levels for trouble tickets
|
Severity 1
|
This level occurs when the business is affected. Everyone should stop working on anything else and focus entirely on fixing this problem as soon as possible. The blast radius, in this case, can be that each company customer is affected. Examples: order drops or a security breach of customer data. SLA can be turnaround of four to six hours.
|
|
Severity 2
|
This level is critical and needs immediate attention, but the blast radius, such as the number of customers affected, may be smaller (with perhaps 40% of customers affected). Example: during the order checkout, the applying code option is broken, so any customer with a discount code cannot use it and hence needs to pay the total price-. This example isn’t a happy customer experience, right? SLA in this case can be 24 hours.
|
|
Severity 3
|
This level is not as critical as the preceding two. On-call support can take one to two days to triage the problem and get back. Example: an application needs to go through a security review before launch. Because the service has not been launched, no production effect occurs, so the SLA is two business days.
|
|
Severity 4
|
This level is not urgent and can wait. It has the least priority for the on-call team and will be addressed when trouble tickets of higher priority are addressed. Example: a marketing manager requests access to your internal platform to show sale numbers. In this case, SLA can be in days, such as five business days.
|
Use the severity table to compare incident impact, affected customers, urgency, and response commitments. section 10.1.5
