MIT AI Risk Repository · Risk Sub-Category · 24.11.03

Weaponised misinformation agents

Category: Misinformation risks

Open the MIT Risk Navigator All entries from Gabriel2024

Description

"Finally, AI assistants themselves could become weaponised by malicious actors to sow misinformation and manipulate public opinion at scale. Studies show that spreaders of disinformation tend to privilege quantity over quality of messaging, flooding online spaces repeatedly with misleading content to sow ‘seeds of doubt’ (Hassoun et al., 2023). Research on the ‘continued influence effect’ also shows that repeatedly being exposed to false information is more likely to influence someone’s thoughts than a single exposure. Studies show, for example, that repeated exposure to false information make

From The Ethics of Advanced AI Assistants (Gabriel2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 4. Malicious actors
Subdomain: 4.1 Disinformation, surveillance, and influence at scale
Causal entity: Human
Intent: Intentional
Timing: Post-deployment

Subdomain definition: Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim to manipulate political processes, public opinion and behavior.

Real-world incidents in this subdomain

Browse all incidents in this subdomain

How other frameworks describe this risk

Coercion/manipulation
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Electoral interference
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Political manipulation
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Political and Economic
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Biased influence through citizen screening and tailored propaganda
The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks (Allianz2018)
Political
The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks (Allianz2018)
Surveillance and Censorship
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Disinformation and manipulation of public opinion
International Scientific Report on the Safety of Advanced AI (Bengio2024)

Other entries from Gabriel2024

Capability failures Risk Category · 7.3
Lack of capability for task Risk Sub-Category · 7.3
Difficult to develop metrics for evaluating benefits or harms caused by AI assistants Risk Sub-Category · 6.5
Safe exploration problem with widely deployed AI assistants Risk Sub-Category · 7.3
Goal-related failures Risk Category · 7.1
Misaligned consequentialist reasoning Risk Sub-Category · 7.3
Specification gaming Risk Sub-Category · 7.1
Goal misgeneralisation Risk Sub-Category · 7.1
Deceptive alignment Risk Sub-Category · 7.1
Malicious Uses Risk Category · 4.0
Offensive Cyber Operations (General) Risk Sub-Category · 4.2
AI-Powered Spear-Phishing at Scale Risk Sub-Category · 4.1