liveBook · Manning

Reconstruction attack

As explained in AI Governance

An attack that attempts to recover original text or other source information from vector representations.

A reconstruction attack attempts to recover snippets of original text from vector representations such as embeddings. In retrieval-augmented generation (RAG), those embeddings may be stored in a vector store and used to retrieve passages. They should not be assumed anonymous: recovery may be feasible in some settings. section 5.3.2

RAG privacy risk

A system embeds customer-service logs for retrieval. An attacker who accesses the vectors may try to recover snippets of the original logs; sensitive-source embeddings should be treated as personal data. section 5.3.2

Classify embeddings by source and whether they can be linked to people. Treat embeddings from sensitive sources—such as patient records, HR files, or customer-service logs—as personal data. section 5.3.2

How AI Governance covers it

How deep it goes
Defines the attack and its privacy implications but does not detail attack implementation.
Examples
Uses RAG embeddings from policies, interaction logs, chat logs, patient records, HR files, and customer-service logs.
What you can do after
Evaluate embedding privacy risk by classifying data sources and checking whether vectors can be linked to people.
How it connects
Links reconstruction attacks to embeddings, vector stores, retrieval-augmented generation, and attribute inference.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.