A model can retain specific details from its training data and reproduce them in response to a prompt. This memorization creates privacy and intellectual-property risks when outputs reveal personal records, confidential material, or copyrighted content. section 5.3.1 section 6.6.3
Unlike a database record, a learned detail may be embedded in model weights rather than stored in an easily deleted location. Repeated training images are particularly likely to be memorized; carefully crafted prompts can sometimes extract or closely reconstruct them. Recognizable resemblance alone, however, is not proof of an exact copy. section 5.1 section 5.3.1
A claim that resurfaced
ClaimAssist included a claimant’s name in a routine response. Investigation found her claim both in a retrieval index without a retention limit and in a fine-tuning dataset. A red-team engineer later extracted her phone number by prompting with accident details. The case required investigating both stored retrieval data and learned model behavior. section 5.3
Memorization is distinct from other ways data persists: logs, backups, embeddings, retrieval indexes, and vendor systems may hold copies even when the model has not learned them. An output that reproduces retrieved material can also raise licensing concerns, so investigation should trace both training and retrieval paths. section 5.3 section 6.6.3
Figure 5.1 Where personal data can live in a GenAI system. The exact boxes depend on the adoption model, but access, correction, deletion, and retention controls rarely stop at the main application database.

Use the data-location map to scope access, retention, and deletion checks beyond the application database. section 5.3
Reduce and test the exposure across the data lifecycle:
- Collect only data needed for the purpose, and prevent personal data from entering training when possible. section 5.2 section 5.3.4
- Test models for memorization, including adversarial attempts to elicit sensitive training details. section 5.3.4 section 5.3
- Trace copies through fine-tuning datasets, model artifacts, retrieval stores, logs, backups, and vendors; maintain auditable pipelines and stress-test deletion. section 5.3 section 5.1
- Review licensing for material supplied through retrieval or fine-tuning; provider safeguards for base-model data may not cover it. section 6.6.3





