MIT AI Risk Repository · Risk Sub-Category · 58.03.04
Coercion/manipulation
Category: Psychological
Description
"Coercion/manipulation - Use of a technology system to covertly alter user beliefs and behaviour using nudging, dark patterns and/or other opaque techniques, resulting in potential erosion of privacy, addiction, anxiety/distress, etc."
From A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Domain
- 4. Malicious actors
- Causal entity
- Human
- Intent
- Intentional
- Timing
- Post-deployment
Subdomain definition: Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim to manipulate political processes, public opinion and behavior.
Real-world incidents in this subdomain
- Suspected AI-Generated Deepfake Video Reportedly Targeted Former Chhattisgarh Chief Minister Bhupesh Baghel on Instagram
- Purportedly AI-Manipulated Satellite Image Reportedly Claimed Iranian Strike Destroyed U.S. Radar in Qatar
- Purported AI-Generated War Footage Reportedly Circulated Widely Online During the Opening Phase of the War in Iran
- Network of Allegedly Fake Facebook Profiles with Purportedly AI-Generated Images Amplified Posts by Bulgaria's 'There Is Such a People' (ITN) Party
- Purportedly AI-Generated Image Reportedly Circulated Ahead of Thai Election Depicting PM Anutin Charnvirakul Dining with Benjamin Mauerberger
- Purportedly AI-Altered Images Reportedly Distort Evidence After Minneapolis Shooting of ICU Nurse Alex Pretti
How other frameworks describe this risk
- Biased influence through citizen screening and tailored propaganda
- Political
- Surveillance and Censorship
- Disinformation and manipulation of public opinion
- Manipulation of public opinion
- AI is used to scale up production of false and misleading information
- AI's persuasive capabilities are misused to gain influence and promote harmful ideologies
- Developments in AI enable actors to undermine democratic processes
Other entries from Abercrombie2024
- Autonomy
- Autonomy/agency loss
- Impersonation/identity theft
- IP/copyright loss
- Personality rights loss
- Physical
- Bodily Injury
- Loss of Life
- Personal Health Deterioration
- Property Damage
- "Psychological - Direct or indirect impairment of the emotional and psychological mental health of an individual, organisation, or society."
- Addiction