Layered safeguards

As explained in Delivering Machine Learning Projects: From design to deployment

Layered safeguards are a safety pattern that combines multiple independent checks and controls to reduce the likelihood and impact of system failures.

Layered safeguards combine checks and controls so that more system mistakes are caught before and after deployment. They connect technical assurance with governance and human control. section 2.6

Build the safety case across these layers:

• External safety or deployment hazards: identify failure causes through a risk register, explicit evaluations, and user reviews. section 2.6
• Monitoring: inspect and record system behavior, then define alerts and notifications for users and stakeholders. section 2.6
• Robustness: characterize and test behavior in relevant circumstances as part of service acceptance. section 2.6
• Alignment: document who controls the system and how humans steer, control, and receive reports about it. section 2.6
• Record who may be harmed when the system works as intended and when it fails. section 2.6

Figure 2.3 A layered model of ML safety (adapted from Hendrycks et al [7])

Use the layered safety figure to review safeguards from deployment hazards through monitoring, robustness, and alignment. section 2.6

How Delivering Machine Learning Projects: From design to deployment covers it

How deep it goes

Explains four safeguard layers and associated practices, but stops short of detailed technical implementation.

Examples

Uses risk registers, explicit evaluations, user reviews, system inspections, alerts, robustness tests, and harm documentation.

What it assumes

Assumes systematic development and stakeholder review while teaching the safeguard layers and their safety rationale.

What you can do after

Apply it via risk registers, evaluations, monitoring procedures, acceptance tests, and documented human controls.

How it connects

Connects risk registers, impact assessment, monitoring, robustness, evaluation, alignment, and human control.

liveBookmanning.com

Read Delivering Machine Learning Projects: From design to deployment on liveBook Get the book at manning.com

Related concepts

• robustnesshas part · Robustness is the ability of a system or model to maintain acceptable behavior or performance despite variations, disturbances, faults, or challenging conditions.
• monitoringhas part · Monitoring is the ongoing observation and measurement of a system’s behavior, state, or operational signals to assess its condition and detect changes or problems.
• alignmenthas part · Alignment is the condition or process of arranging, adjusting, or coordinating elements so they correspond to one another or meet a reference, constraint, or objective.

This explainer was generated by Manning's knowledge graph from the text of Delivering Machine Learning Projects: From design to deployment. The section links point to the parts of the book it draws on.