Privacy protects people and their information from unwanted observation, collection, use, or disclosure. In generative AI, risks appear throughout the pipeline: models may memorize training data, while users may reveal sensitive information through prompts. section 10.9 section 3.20
Privacy risks and responses include:
- Training data can contain personally identifiable information (PII), private messages, medical details, contact information, or financial numbers that models may reproduce. section 10.18 section 10.9
- User prompts and chat histories can expose confidential information, particularly when people assume that conversational systems keep their inputs private. section 3.10 section 10.9
- Organizations respond with acceptable-use policies, privacy assessments, internal review, consent frameworks, anonymization, pseudonymization, and differential privacy. section 3.17 section 10.18
- These safeguards reduce risk but do not guarantee privacy, because reidentification, imperfect detection, inference, and model leakage remain possible. section 3.17 section 3.19
A confidential contract prompt
An attorney enters a client’s draft contract into a dialogue agent for revisions. The tool may save time, but the prompt could expose confidential information to trainers, future model use, or other users, and could breach attorney-client privilege without consent. section 3.10
Privacy is therefore both a technical and governance problem. Laws and regulators address lawful collection, storage, processing, and disposal, while developers and users must make careful choices about training data, permissions, prompts, and disclosure. section 3.13 section 3.19
