Model memorization is the retention of specific training examples or sensitive information in a model’s behavior. Large language models (LLMs) can unintentionally reproduce personally identifiable information (PII), including names, addresses, phone numbers, and other private details, even when trained on public data. The risk increases when proprietary or sensitive datasets are exposed through public deployment. section 3.4
Memorization creates several practical and governance concerns:
- Training-data extraction attacks use systematic queries to recover identifiable information from a model. section 3.4
- A 2021 GPT-2 demonstration recovered hundreds of memorized sequences, including contact details, conversations, code, and universally unique identifiers; some appeared only once in training data. section 3.4
- Larger models showed greater susceptibility in that demonstration, while later research found that existing safety techniques do not fully prevent leakage. section 3.4
