liveBook · Manning

Differential privacy

As explained in AI Governance

Differential privacy is a mathematical framework that limits how much an individual’s data can affect an analysis or model output.

A generative model can expose personal information by reproducing or enabling reconstruction of its training data. Differential privacy limits how much any one person’s data can affect the model’s outputs, providing a mathematical privacy guarantee. section 5.2.2

During training or fine-tuning, differentially private noise introduces controlled randomness so the model can learn general patterns while limiting the influence of individual records. This protection has a trade-off: noise can reduce accuracy, so teams must decide where lower precision is acceptable. section 5.2.2 section 5.3.2

Use differential privacy as part of a broader privacy workflow:

  • Pseudonymize or filter training data where appropriate, alongside privacy-enhancing training techniques. section 5.2.2
  • Test for known attacks, including data reconstruction and attribute inference, rather than assuming that training safeguards make a model anonymous. section 5.2.2
  • For systems using embeddings, separately protect access to the embedding database and verify that deletion requests propagate to vectors, replicas, and backups. section 5.3.2

Whether a trained model qualifies as anonymous remains a legal question with differing regulatory views. In practice, treat a model that can reproduce or reconstruct personal details as in scope for data protection. section 5.2.2

How AI Governance covers it

How deep it goes
Defines the guarantee and describes noise during training, without specifying an implementation procedure.
Examples
Uses generative models trained on personal data and model fine-tuning to illustrate privacy protection.
What you can do after
Evaluate the privacy–accuracy trade-off when considering noise during model training.
How it connects
Connects differential privacy with anonymization, data reconstruction, attribute inference, and model training.

Also explained in

Compare how 4 books explain differential privacy →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.