A privacy pillar is a dimension for organizing where privacy risks emerge across the generative AI lifecycle. Database and log reviews alone may miss risks in model weights, embedding stores, retrieval pipelines, and generated outputs. section 5.1
Collection and purpose highlights risks in data entering or being reused by AI systems:
- Combining datasets can make people identifiable even when individual sources have been cleaned. section 5.1
- Prompts and responses may be processed for model improvement, creating secondary-use concerns. section 5.1
Training-data extraction
Researchers prompted ChatGPT (gpt-3.5-turbo) to repeat a word until it began outputting training data, including personal details. section 5.1
Table 5.1 Traditional Privacy vs GenAI Privacy
|
Privacy
Topic
|
Traditional
Application
|
GenAI
Application
|
GenAI-specific
Mitigations
|
|
Collection & Purpose
|
Only
necessary fields are collected from the user or re-used from existing
datasets. Lawful basis tied to collection context.
|
Training
sets scraped from the web include people who never dealt with the vendor. A
search engine listing can be removed on request; text absorbed into model
weights cannot. Vendors may process user data by default for model
improvement.
|
Input
filters: anonymize or pseudonymize finetune/RAG data, apply strict retention
policies, tell users how their data is used (e.g. to give AI-generated
responses, train certain models) and offer opt-outs or opt-ins, opt-out of
vendor logging/training of prompts and data.
|
|
Storage & Memorization
|
Records live
in a database; deleting means removing a row.
|
Personal
details can get “baked in” as patterns in model weights, embeddings, or
vector databases.Models can spit out memorized names or documents; embeddings
reveal hidden details
|
Encrypt and
lock down embeddings/vector DBs, delete logs after a short period, test
models for leakage
|
|
Output Integrity
|
Software
gives fixed results you can trace to inputs. Data leakage typically results
from misconfigurations or insider access. ML model misclassifications can be
debugged.
|
Models can
make up personal facts and false claims about people (defamation)
non-deterministically (e.g. make guesses of user traits from context),
can disclose sensitive data without being asked, LLM is a black box and
debugging is difficult.
|
Output
filters, isolate project/session, red-team for personal data leaks and
hallucinations
|
|
User Rights & Governance
|
Users are
offered tools to see, correct, or delete their data.
|
Hard to
erase or fix info once it’s spread across billions of model parameters.
|
Reduce
sensitive data in training (E.g. differential privacy/anonymization), clear
opt-outs from model training, vendor contracts with delete options
(retention, deletion upon ending the contract, delete if a user leaves the
organization, etc.), model retraining, Machine unlearning (once it becomes
more mature)
|
Compare traditional privacy practices with generative AI risks and collection-focused mitigations. section 5.1
