A membership inference attack probes a model to determine whether a particular person’s data was used in training. Membership can itself be sensitive: inclusion in a cancer registry or bankruptcy dataset may reveal information about the person, even if the model does not reproduce a record verbatim. section 5.3.1 section 5.2.2
An attacker asks indirect questions and looks for differences between the model’s responses to real facts and made-up ones. Those differences can provide clues about training-set membership. This differs from memorization, where training data reappears in an output, and from attribute inference, where a model predicts hidden traits from text. section 5.3.1
Sensitive registry membership
If a model was trained on a cancer registry, an attacker who infers that a person’s data was included may learn a sensitive fact about that person. The privacy failure does not require the model to disclose the person’s full record. section 5.3.1 section 5.2.2
Reduce exposure across the training and deployment lifecycle:
- Filter sensitive material, remove duplicates, and limit access before using data for training. section 5.3.1
- Where personal data is used for fine-tuning, account for membership inference in the model and actively test for memorization; prevent personal data from entering training when possible. section 5.3.4
- Assess anonymity using measures such as filtering or pseudonymization, privacy-enhancing techniques during training, and testing for known attacks. Removing direct identifiers alone is not foolproof. section 5.2.2

