MIT AI Risk Repository · domain 4: Malicious actors
4.3 Fraud, scams, and targeted manipulation
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
- 77
- 12
- 412
- 402
| Label | Value |
|---|---|
| Human | 62 |
| Other | 10 |
| AI | 5 |
| Label | Value |
|---|---|
| Intentional | 63 |
| Other | 13 |
| Unintentional | 1 |
| Label | Value |
|---|---|
| Post-deployment | 72 |
| Other | 5 |
| Label | Value |
|---|---|
| 2014 | 1 |
| 2016 | 2 |
| 2017 | 2 |
| 2018 | 1 |
| 2019 | 3 |
| 2020 | 9 |
| 2021 | 4 |
| 2022 | 13 |
| 2023 | 43 |
| 2024 | 103 |
| 2025 | 190 |
| 2026 | 40 |
| Label | Value |
|---|---|
| Risk Category | 13 |
| Risk Sub-Category | 64 |
Risk entries
Browse and export all- Impersonation/identity theft
"Impersonation/identity theft - Theft of an individual, group or organisation’s identity by a third-party in order to defraud, mock or otherwise harm them."
- IP/copyright loss
"IP/copyright loss - Misuse or abuse of an individual or organisation’s intellectual property, including copyright, trademarks, and patents."
- Dehumanisation/objectification
"Dehumanisation/objectification - Use or misuse of a technology system to depict and/or treat people as not human, less than human, or as objects."
- Defamation/libel/slander
"Defamation/libel/slander - Use of a technology system to create, facilitate or amplify false perception(s) about an individual, group, or organisation."
- Financial and business
"Financial and Business - Use or misuse of a technology system in a manner that damages the financial interests of an individual or group, or which causes strategic, operational, legal or financial ha...
- Cheating/plagiarism
"Cheating/plagiarism - Use of another person’s or group’s words or ideas without consent and/or acknowledgement."
- Misinformation and Manipulation
"Recent studies have demonstrated that LLMs can be exploited to craft deceptive narratives with levels of persuasiveness similar to human-generated content (Pan et al., 2023b; Spitale et al., 2023), t...
- Cybersecurity
"LLMs may exacerbate cybersecurity risks in various ways (Newman, 2024). Firstly, LLMs may significantly amplify the effectiveness of deceptive operations aimed at tricking people into disclosing sens...
- Domain-Specific Misuses
"Improvements in LLMs may exert greater pressure to apply LLMs to various domains, such as health and education (Eloundou et al., 2023). Crude efforts to use LLMs in such domains, however, may incur h...
- Harm to individuals through fake content
"General- purpose AI systems can be used to increase the scale and sophistication of scams and fraud, for example through general- purpose AI- enhanced ‘phishing’ attacks. General- purpose AI can be u...
- Harm to individuals through fake content
"Malicious actors can use general- purpose AI to generate fake content that harms individuals in a targeted way. For example, they can use such fake content for scams, extortion, psychological manipul...
- Unhelpful Uses
"Improper uses of LLM systems can cause adverse social impacts."
- Academic Misconduct
"Improper use of LLM systems (i.e., abuse of LLM systems) will cause adverse social impacts, such as academic misconduct."
- Scams
"Bad actors can also use generative AI tools to produce adaptable content designed to support a campaign, political agenda, or hateful position and spread that information quickly and inexpensively ac...
- Harassment, Impersonation, and Extortion
"Deepfakes and other AI-generated content can be used to facilitate or exacerbate many of the harms listed throughout this report, but this section focuses on one subset: intentional, targeted abuse o...
- Malicious intent
"A frequent malicious use case of generative AI to harm, humiliate, or sexualize another person involves generating deepfakes of nonconsensual sexual imagery or videos."
- Privacy and consent
"Even when a victim of targeted, AIgenerated harms successfully identifies a deepfake creator with malicious intent, they may still struggle to redress many harms because the generated image or video...
- Believability
Deepfakes can impose real social injuries on their subjects when they are circulated to viewers who think they are real. Even when a deepfake is debunked, it can have a persistent negative impact on h...
- Data Security Risk
"Just as every other type of individual and organization has explored possible use cases for generative AI products, so too have malicious actors. This could take the form of facilitating or scaling u...
- Deception - Synthetic identities
"GenAI can produce images of people that look very real, as if they could be seen on platforms like Facebook, Twitter, or Tinder. Although these individuals do not exist in reality, these synthetic id...
- Propaganda - Digital impersonations
"AI-generated impersonation for identity theft might be found at the intersection of “Harm to the Person” and “Deception.”"
- Dishonesty - Targeted harassment
"LLMs can be deployed to target individuals online, sending them personalized and harmful messages at scale"
- Financial and Economic Damage
"Then, we have the potential for financial loss, fraud, market manipulation, and other economic harms, which fall under “Financial and Economic Damage.”
- Deception - Bespoke ransom
-
- Dishonesty - Market manipulation
-