MIT AI Risk Repository · domain 4: Malicious actors
4.1 Disinformation, surveillance, and influence at scale
Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim to manipulate political processes, public opinion and behavior.
- 84
- 12
- 138
- 132
| Label | Value |
|---|---|
| Human | 59 |
| Other | 14 |
| AI | 9 |
| Not coded | 2 |
| Label | Value |
|---|---|
| Intentional | 73 |
| Other | 9 |
| Not coded | 2 |
| Label | Value |
|---|---|
| Post-deployment | 74 |
| Other | 8 |
| Not coded | 2 |
| Label | Value |
|---|---|
| 2015 | 2 |
| 2016 | 1 |
| 2017 | 1 |
| 2019 | 2 |
| 2020 | 6 |
| 2021 | 3 |
| 2022 | 6 |
| 2023 | 21 |
| 2024 | 50 |
| 2025 | 38 |
| 2026 | 8 |
| Label | Value |
|---|---|
| Risk Category | 10 |
| Risk Sub-Category | 74 |
Risk entries
Browse and export all- Authoritarian Surveillance, Censorship, and Use (General)
"While new technologies like advanced AI assistants can aid in the production and dissemination of decision-guiding information, they can also enable and exacerbate threats to production and dissemina...
- Authoritarian Surveillance, Censorship, and Use: Authoritarian Surveillance and Targeting of Citizens
"Authoritarian governments could misuse AI to improve the efficacy of repressive domestic surveillance campaigns. Malicious actors will recognize the power of AI targeting tools. AI-powered analytics...
- Authoritarian Surveillance, Censorship, and Use: Delegation of Decision-Making Authority to Malicious Actors
"Finally, the principal value proposition of AI assistants is that they can either enhance or automate decision-making capabilities of people in society, thus lowering the cost and increasing the accu...
- Weaponised misinformation agents
"Finally, AI assistants themselves could become weaponised by malicious actors to sow misinformation and manipulate public opinion at scale. Studies show that spreaders of disinformation tend to privi...
- Driving opinion manipulation
"AI assistants may facilitate large-scale disinformation campaigns by offering novel, covert ways for propagandists to manipulate public opinion. This could undermine the democratic process by distort...
- Deception (e.g., fraud)
- Manipulation (e.g., deepfakes)
- Automatically generating disinformation at scale
"Disinformation (in various modalities: text, audio, images, video, etc.) can be generated with minimal human oversight and effort. Disinformation tools are relatively cheap and their technology is wi...
- Generative AI use in political influence campaigns
"GPAI tools can be used in automation and scaling of influence campaigns [178]. Public opinion may be manipulated by targeted misleading or manipulative information. This can lead to rising political...
- Misuse for surveillance and population control
"AI tools can be misused by human or institutional actors for monitoring, control- ling, or suppressing individuals [178]. Massive data collection and automated analysis are often conducted, and AI to...
- Diminishing societal trust due to disinformation or manipulation
"The use of GPAIs may contribute to the proliferation of either deliberate dis- information or unintended misinformation can severely erode trust in public figures and democratic institutions. This di...
- Personalized disinformation
"Automatic generation of disinformation can be personalized to target specific groups or individuals. Such attacks can be more effective in achieving their goals, and their costs can be significantly...
- Malicious use and abuse (mass surveillance)
"Generative AI facilitates the automation of data analysis, offering numerous benefits, such as increased speed and the ability to process large volumes of information efficiently. Such ability signif...
- Misinformation and disinformation
"IIl-intentioned individuals or entities may deliberately use generative AI models to produce and spread disinformation—false or misleading information knowingly presented as if true—on a massive scal...
- Society Manipulation
manipulation of social dynamics
- Deepfake Technology
AI employed to produce convincing counterfeit visuals, videos, and audio clips that give the impression of authenticity
- Persuasive AIs
"The deliberate propagation of disinformation is already a serious issue, reducing our shared understanding of reality and polarizing opinions. AIs could be used to severely exacerbate this problem by...
- Manipulation
"The 2016 scandal involving Cambridge Analytica is the most infamous example where people's data was crawled from Facebook and analytics were then provided to target these people with manipulative con...
- Spreading disinformation
"Generative AI models might be used to intentionally create misleading or false information to deceive or influence a targeted audience."
- Persuasion and manipulation
"These evaluations seek to ascertain the effectiveness of a LLM in shaping people's beliefs, propagating specific viewpoints, and convincing individuals to undertake activities they might otherwise av...
- Political Strategy
"LLM can take into account rich social context and undertake the necessary social modelling and planning for an actor to gain and exercise political influence"
- Disinformation
"These evaluations assess a LLM's ability to generate misinformation that can be propagated to deceive, mislead or otherwise influence the behaviour of a target (Liang et al., 2022)."
- Privacy
"OpenAI’s GPT-3 was designed to be dicult to extract personal information from, including for example public gures’ dates of birth. Even so, malicious uses of AI continue to encroach on privacy, as ex...
- Institutional trust loss
"Erosion of trust in public institutions and weakened checks and balances due to mis/disinformation, influence operations, or real or perceived misuse of generative AI"
- Coercion / manipulation
"Use of a technology system to covertly alter user beliefs and behaviour using nudging, dark patterns and/or other opaque techniques"