liveBook · Manning

Storage and memorization

As explained in AI Governance

Storage and memorization is a privacy-risk dimension concerning how personal information persists in AI models and associated data stores.

Personal data can persist after a prompt or dataset has been used. Unlike a conventional database record, it may be encoded in model weights, embeddings, or retrieval indexes, making deletion difficult to verify. section 5.1

Check the system’s full data path for:

  • Training data that a model may reproduce verbatim. section 5.1
  • Embedding databases and retrieval indexes that may retain or expose personal details. section 5.1
  • Prompts, responses, and backups across vendor systems and organizational infrastructure. section 5.0

Memorized training data

Researchers prompted a model to repeat a word hundreds of times and extracted more than 10,000 verbatim text fragments, some containing names and contact details. section 5.1

Treat deletion as a verification problem, not just a record-removal task. Maintain auditable data pipelines and stress-test whether personal information persists in less visible layers. Controls and evidence should account for SaaS, API integrations, and self-hosted models. section 5.0 section 5.1

Table 5.1 Traditional Privacy vs GenAI Privacy
Privacy Topic
Traditional Application
GenAI Application
GenAI-specific Mitigations
Collection & Purpose
Only necessary fields are collected from the user or re-used from existing datasets. Lawful basis tied to collection context.
Training sets scraped from the web include people who never dealt with the vendor. A search engine listing can be removed on request; text absorbed into model weights cannot. Vendors may process user data by default for model improvement.
Input filters: anonymize or pseudonymize finetune/RAG data, apply strict retention policies, tell users how their data is used (e.g. to give AI-generated responses, train certain models) and offer opt-outs or opt-ins, opt-out of vendor logging/training of prompts and data.
Storage & Memorization
Records live in a database; deleting means removing a row.
Personal details can get “baked in” as patterns in model weights, embeddings, or vector databases.Models can spit out memorized names or documents; embeddings reveal hidden details
Encrypt and lock down embeddings/vector DBs, delete logs after a short period, test models for leakage
Output Integrity
Software gives fixed results you can trace to inputs. Data leakage typically results from misconfigurations or insider access. ML model misclassifications can be debugged.
Models can make up personal facts and false claims about people (defamation) non-deterministically (e.g. make guesses of user traits from context), can disclose sensitive data without being asked, LLM is a black box and debugging is difficult.
Output filters, isolate project/session, red-team for personal data leaks and hallucinations
User Rights & Governance
Users are offered tools to see, correct, or delete their data.
Hard to erase or fix info once it’s spread across billions of model parameters.
Reduce sensitive data in training (E.g. differential privacy/anonymization), clear opt-outs from model training, vendor contracts with delete options (retention, deletion upon ending the contract, delete if a user leaves the organization, etc.), model retraining, Machine unlearning (once it becomes more mature)

Compare conventional record deletion with GenAI persistence in model weights and other data stores. section 5.1

How AI Governance covers it

How deep it goes
Explains persistence in weights, embeddings, and indexes, then calls for auditable pipelines and stress tests to verify deletion.
Examples
Uses repeated-word prompting to extract training snippets and similarity searches to expose identities from embeddings.
What you can do after
Diagnose persistence through stress tests and auditable data pipelines.
How it connects
Connects privacy storage to model weights, retrieval-augmented generation, embeddings, deletion, and user rights.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.