MIT AI Risk Repository · domain 4: Malicious actors
4.1 Disinformation, surveillance, and influence at scale
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim to manipulate political processes, public opinion and behavior.
- 84
- 12
- 138
- 132
| Label | Value |
|---|---|
| Human | 59 |
| Other | 14 |
| AI | 9 |
| Not coded | 2 |
| Label | Value |
|---|---|
| Intentional | 73 |
| Other | 9 |
| Not coded | 2 |
| Label | Value |
|---|---|
| Post-deployment | 74 |
| Other | 8 |
| Not coded | 2 |
| Label | Value |
|---|---|
| 2015 | 2 |
| 2016 | 1 |
| 2017 | 1 |
| 2019 | 2 |
| 2020 | 6 |
| 2021 | 3 |
| 2022 | 6 |
| 2023 | 21 |
| 2024 | 50 |
| 2025 | 38 |
| 2026 | 8 |
| Label | Value |
|---|---|
| Risk Category | 10 |
| Risk Sub-Category | 74 |
Risk entries
Browse and export all- Propaganda
LLMs can be leveraged, by malicious users, to proactively generate propaganda information that can facilitate the spreading of a target
- Politically motivated misuse
"General purpose AI models could exacerbate existing tactics for political destabilisation, such as disinformation campaigns, and surveillance efforts if misused for political motivations. The technol...
- Sockpuppeting
"Create synthetic online personas or accounts"
- Falisification
"Fabricate or falsely represent evidence, incl. reports, IDs, documents"
- Information Integrity
"Lowered barrier to entry to generate and support the exchange and consumption of content which may not distinguish fact from opinion or fiction or acknowledge uncertainties, or could be leveraged for...
- Civic and political harms
Political harms emerge when “people are disenfranchised and deprived of appropriate political power and influence” [186, p. 162]. These harms focus on the domain of government, and focus on how algori...
- Other ethical risks
"Although we have discussed a number of common risks posed by ML systems, we acknowledge that there are many other ethical risks such as the potential for psychological manipulation, dehumanization, a...
- Cognitive risks (Risks of usage in launching cognitive warfare)
"AI can be used to make and spread fake news, images, audio, and videos; propagate content of terrorism, extremism, and organized crimes; interfere in the internal affairs of other countries, social s...
- Manipulation
"The predictability of behaviour protocol in AI, particularly in some applications, can act an incentive to manipulate these systems."
- Large-Scale Persuasion and Harmful Manipulation Risks
"AI systems can be gravely misused to distort public perception and compromise social stability through the generation of synthetic content (e.g., deepfakes, sophisticated fake news) and the strategic...
- Information
"Large-scale influence on communication and information systems, and epistemic processes more generally."
- Ability to persuade
"AI could be used to develop sophisticated tools to manipulate and persuade individuals."
- Advertising-driven models
"AI models and systems underpin the advertising approaches that drive much of the internet, potentially influencing societal behavior."
- AI in totalitarian regimes
"AI-based surveillance and manipulation could be used to maintain global totalitarian regimes."
- Limitations in model generative accuracy
"AI-generated deepfakes can create convincingly realistic but entirely fabricated information."
- Surveillance capabilities
"AI models and systems may grant governments or corporations increased monitoring over individuals."
- Widespread use of persuasion tools
"Widespread use of AI-powered persuasion tools could lead to systemic harm"
- Elections
"This category addresses responses that contain factually incorrect information about electoral systems and processes, including in the time, place, or manner of voting in civic elections."
- Making disinformation cheaper and more effective
"LMs can be used to create synthetic media and ‘fake news’, and may reduce the cost of producing disinformation at scale (Buchanan et al., 2021). While some predict that it will be cheaper to hire hum...
- Illegitimate surveillance and censorship
"The collection of large amounts of information about people for the purpose of mass surveillance has raised ethical and social concerns, including risk of censorship and of undermining public discour...
- Making disinformation cheaper and more effective
"While some predict that it will remain cheaper to hire humans to generate disinformation [180], it is equally possible that LM- assisted content generation may offer a lower-cost way of creating disi...
- Illegitimate surveillance and censorship
Anticipated risk: "Mass surveillance previously required millions of human analysts [83], but is increasingly being automated using machine learning tools [7, 168]. The collection and analysis of larg...
- Influence operations
"Facilitating large-scale disinformation campaigns and targeted manipulation of public opinion"
- Defamation
"Facilitating slander, defamation, or false accusations"
- Privacy and safety
"Privacy and safety deals with the challenge of protecting the human right for privacy and the necessary steps to secure individual data from unauthorized external access. Many organizations employ AI...