chapter five

5 Privacy

 

This chapter covers

  • When GenAI privacy fails in practice.
  • Four pillars of data protection in GenAI
  • Practical steps to reduce risks in each pillar
  • How deployment posture changes privacy risks
  • What evidence regulators look for

Privacy is often equated with security, but good security is not sufficient for good privacy. Encrypting data and locking it behind access controls keeps it safe from data breaches. Privacy therefore requires good security (confidentiality and integrity are essential. However, privacy goes further. It defines the rules for collection, use, storage, sharing, and deletion, and it makes sure individuals remain in control. It governs what you are allowed to do with the data even in a secure environment. It is about lawfulness, fairness, and control for the individual.

Good privacy often requires (among others):

5.1 Collection and Purpose

5.1.1 No Valid Legal Basis

5.1.2 Overcollection

5.1.3 Vendor’s Inappropriate Use of Organizational Data

5.2 Storage and Memorization

5.2.1 Memorization and Membership Inference

5.2.2 Insufficient deletion of embeddings

5.2.3 Insufficient Deletion of (Meta)data

5.3 Output Integrity

5.3.1 Hallucinations and Defamations

5.3.2 Overreliance and Automated Decision Making

5.4 User Rights & Governance

5.4.1 Making models forget and correct

5.4.2 Data Subject Access Requests (DSAR)

5.4.3 Transparency

5.5 Adoption Models: Where to Focus

5.5.1 SaaS consumers

5.5.2 API integrators

5.5.3 Model hosters

5.5.4 Top risks

5.5.5 What counts as evidence?

5.6 Summary