A name in a public post, a student’s assignment, and a private message entered into a chatbot can all raise privacy questions. Personal data is information relating to an identified or identifiable person. It includes direct identifiers, such as names and contact details, and information that can identify someone indirectly. Public availability does not necessarily remove privacy protections. section 3.18 section 3.15 section 3.16
Large language models (LLMs) encounter personal data in two main ways: through the data used to train them and through information people enter while using them. Personally identifiable information (PII) in training data may be memorized and reproduced in an answer. A conversational service may also retain user inputs, make them available to reviewers, or use them to improve the system. What happens depends on the provider’s policies and controls. section 3.4 section 3.11
Why public training data can still matter
In a 2021 demonstration, researchers extracted memorized sequences from GPT-2, a model trained on publicly available data. The recovered material included names, phone numbers, email addresses, and conversations; some sequences had appeared only once in the training data. The result illustrates a risk, not a claim that every prompt will reveal personal data. section 3.4
Reducing exposure requires choices throughout a system’s life:
- Limit training on sources known to contain substantial PII, and consider techniques such as anonymization or differential privacy. Removing obvious identifiers is imperfect: other details may still allow a person to be identified. section 10.18 section 3.17
- Avoid entering information into a chatbot that should not be retained or seen by a reviewer. Check the service’s current controls rather than assuming a conversation is private or that an individual prompt can be deleted. section 3.11
- Set rules for sensitive uses. In education, uploading student assignments or personal details to an outside AI service may expose protected records; schools must consider their obligations under the Family Educational Rights and Privacy Act (FERPA). section 3.16
Privacy is also a legal and governance question. The European Union’s General Data Protection Regulation (GDPR) governs the collection, storage, and use of personal data and requires a lawful basis for processing it. Its protections can apply even to information posted publicly. Yet AI-generated inferences, such as estimates of income or personality, present difficult questions in practice: they can affect people even when oversight or recourse is limited. Technical safeguards and legal compliance are therefore starting points, not guarantees against harm. section 3.15 section 3.18
