Privacy governs whether data can be sourced, accessed, used, and disclosed appropriately in an ML project. Unknown personal-data origins or missing consent can make intended use unethical or illegal, expose the organization to liability, and halt delivery. section 2.0 section 2.8
Make privacy an explicit pre-project and model-selection responsibility:
- Identify privacy stakeholders for every core dataset, relevant organization, and IT platform. section 2.5
- Establish the data privacy requirements and processes that must be negotiated, initially at a top level. section 2.5
- Record unresolved privacy obstacles in the project risk register and assess whether they affect contractual assumptions, viability, or cost. section 2.5
- Treat privacy as a red-line selection measure: reject models that leak information or permit personal data to be extracted. section 8.3.4
Testing a language model for disclosure
A language model trained on medical cases can be probed with a prompt resembling a named person’s medical record. If it completes the prompt with private information, the model creates potential harm and may breach data-protection laws. section 8.3.4
Figure 2.2 An example of a security organization within a large company. Each market-facing unit owns the security relevant to that line of business. A group of marketing units also has a security function, which reports to the CFO, and an IT security unit that reports to the CIO or CTO.
Map privacy ownership alongside security responsibility across market-facing business units, group functions, and IT platforms. section 2.5
