liveBook · Manning

Impact assessment

As explained in Delivering Machine Learning Projects: From design to deployment

An impact assessment is a structured evaluation of a system's intended and unintended effects, costs, benefits, and risks on affected parties.

An impact assessment is a structured check of who may be affected by a proposed ML system, what benefits and costs they may experience, and what ethical issues deployment may create. Begin once the project hypothesis is clear: document corporate-responsibility and ethical requirements, challenge the project assumptions, and create privacy and data impact assessment (PDIA) and algorithmic impact assessment (AIA) documents. section 2.6 section 6.2

Make the assessment a project checkpoint, not a one-time form:

  • Involve stakeholders, including users and people affected by the system; examine costs alongside benefits for those affected. section 6.2
  • Consider the full project and system lifecycles. Decide how impacts will be measured as the system evolves and whether its governance mechanisms are adequate. section 6.2
  • Check applicable data and domain-specific law, while treating legal compliance as a floor rather than a complete ethical assessment. section 2.6

Revisit the assessment after exploratory data analysis and further work on business requirements. The stronger understanding gained at that point supports a better review of ethical, privacy, and security issues. Available evaluation tools vary in project-stage fit and ethical scope, so choose deliberately rather than assuming any one tool is complete. section 6.2

How Delivering Machine Learning Projects: From design to deployment covers it

How deep it goes
Specifies stakeholders, costs, benefits, lifecycle, measurement, and governance, but does not prescribe a complete assessment methodology.
Examples
Uses PDIA and AIA documents and an EDA ethics-check ticket as concrete project artifacts.
What it assumes
Assumes a clear project hypothesis for the initial check and improved data and business understanding for the later review.
What you can do after
Practice/perform it by documenting early impacts, then reassessing affected parties and governance when project understanding improves.
How it connects
Links impact assessment to corporate social responsibility, privacy, security, exploratory data analysis, governance, and regulatory compliance.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Delivering Machine Learning Projects: From design to deployment. The section links point to the parts of the book it draws on.