liveBook · Manning

Privacy attack

As explained in AI Governance

A privacy attack is an adversarial method for extracting personal information or determining sensitive facts from data or an AI system.

A privacy attack exploits residual patterns in data or a model to piece together identity clues, even after direct identifiers are removed. Anonymization reduces risk but does not guarantee that personal information cannot be inferred. section 5.2.2

Before releasing a model publicly, test whether its residual patterns can be exploited. Netflix and AOL are cautionary cases: data considered anonymized may still be shown to expose information. section 5.2.2

How AI Governance covers it

How deep it goes
Warns that anonymization may leave identity clues, but defers specific attack techniques and testing methods.
Examples
Cites Netflix and AOL as cautionary cases where supposedly anonymized data may still expose information.
What you can do after
Evaluate a model before public release by testing whether residual patterns remain exploitable.
How it connects
Connects privacy attacks to anonymization, differential privacy, and overcollection.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.