liveBook · Manning

Ethical review

As explained in Delivering Machine Learning Projects: From design to deployment

Ethical evaluation is the systematic assessment of a system's effects, risks, and compliance with ethical considerations.

Ethical review systematically assesses an ML system’s effects, risks, privacy, security, and governance as understanding of the data, requirements, and deployment context improves. Treat it as a recurring delivery checkpoint, not a final approval step: review after initial understanding, during model selection, and before production deployment. section 6.2 section 9.7

Turn the review into explicit project work:

  • Create an ethics-checkpoint ticket and involve users, project stakeholders, and people affected by the system. section 6.2 section 9.7
  • Compare costs and benefits for affected people, define how impact will be measured over the system lifecycle, and document the assessment. section 6.2 section 9.7
  • Check governance, accountability, effective controls, user understanding, system performance, and mechanisms for managing behavior over time. section 9.7
  • Use an appropriate evaluation framework or tool, while recognizing that available approaches differ in project stage, coverage, and completeness. section 6.2

When review stops delivery

If assessment reveals that the desired system would cause unacceptable harm to previously unidentified stakeholders, stop the project rather than optimize an unsafe deployment. Record the finding and use the decision to reduce further delivery risk. section 6.4

Figure 1.5 The structure of the project described in this book; from creating and developing the project through to managing the final models in production.

Place ethical review across the delivery lifecycle, from project creation and development through production management, alongside documentation and governance checkpoints. section 1.4

How Delivering Machine Learning Projects: From design to deployment covers it

How deep it goes
Provides lifecycle requirements and deployment checklists, but defers a complete standardized methodology because available tools remain uneven.
Examples
Uses an ethics-checkpoint ticket, external evaluation tools, accumulated assessment documents, governance questions, and an unacceptable-harm stop decision.
What it assumes
Assumes developing knowledge of data and business requirements, while requiring consultation with users and affected people.
What you can do after
Evaluate deployment consequences, document impacts, establish controls and governance, or stop a system that creates unacceptable harm.
How it connects
Connects impact assessment with stakeholders, privacy, security, data constraints, model selection, governance, accountability, and production management.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Delivering Machine Learning Projects: From design to deployment. The section links point to the parts of the book it draws on.