liveBook · Manning

Manipulation and exploitation of vulnerabilities

As explained in AI Governance

Manipulation and exploitation of vulnerabilities are AI practices that influence people through manipulative tactics or by taking advantage of vulnerabilities such as age, disability, or financial circumstances.

Article 5 of the EU AI Act prohibits AI practices that materially distort a person’s behavior and cause, or are reasonably likely to cause, significant harm. It covers two routes: manipulative techniques and exploitation of vulnerabilities. Manipulation may be subliminal or openly deceptive; visibility does not mean a person can resist it. The technique-based prohibition also requires appreciable impairment of the person’s ability to make an informed decision. section 7.2.2

Assess the practice by asking:

  • Does it use subliminal, purposefully manipulative, or deceptive techniques that distort behavior and meet the harm threshold? section 7.2.2
  • Does it exploit vulnerability due to age, disability, or a specific social or economic situation, with the required distortion and harm? section 7.2.2

Targeting under pressure

An insurance system flags elderly policyholders showing signs of cognitive decline and prompts agents to use urgency-based scripts to sell a complex product. A chatbot that invokes a user’s children to manufacture urgency for a high-interest loan illustrates visible but hard-to-recognize manipulation.

How AI Governance covers it

How deep it goes
Distinguishes two prohibited routes and specifies distortion, harm, and informed-decision thresholds.
Examples
Uses insurance sales to elderly policyholders and a chatbot pushing a high-interest loan through emotional pressure.
What you can do after
Evaluate whether a system’s tactics, target vulnerabilities, behavioral distortion, and likely harm meet the prohibition criteria.
How it connects
Connects the EU AI Act prohibition to subliminal techniques, deceptive tactics, and vulnerability exploitation.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.