MIT AI Risk Repository · domain 4: Malicious actors
4.3 Fraud, scams, and targeted manipulation
Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
- 77
- 12
- 412
- 402
| Label | Value |
|---|---|
| Human | 62 |
| Other | 10 |
| AI | 5 |
| Label | Value |
|---|---|
| Intentional | 63 |
| Other | 13 |
| Unintentional | 1 |
| Label | Value |
|---|---|
| Post-deployment | 72 |
| Other | 5 |
| Label | Value |
|---|---|
| 2014 | 1 |
| 2016 | 2 |
| 2017 | 2 |
| 2018 | 1 |
| 2019 | 3 |
| 2020 | 9 |
| 2021 | 4 |
| 2022 | 13 |
| 2023 | 43 |
| 2024 | 103 |
| 2025 | 190 |
| 2026 | 40 |
| Label | Value |
|---|---|
| Risk Category | 13 |
| Risk Sub-Category | 64 |
Risk entries
Browse and export all- Harmful Content Generation at Scale: Non-Consensual Content
"The misuse of generative AI has been widely recognized in the context of harms caused by non-consensual content generation. Historically, generative adversarial networks (GANs) have been used to gene...
- Harmful Content Generation at Scale: Fraudulent Services
"Malicious actors could leverage advanced AI assistant technology to create deceptive applications and platforms. AI assistants with the ability to produce markup content can assist malicious users wi...
- AI-driven highly personalized advertisement
"Advanced GPAI systems can create advertisements tailored to individual recip- ients, exploiting the biases and irrational beliefs of each recipient. Such adver- tisements can cause consumers to make...
- Multimodal deepfakes
"Deepfakes are media that depict real or non-existent people or events, involving the use of multiple modalities (e.g., images, audio, video). They can also involve the imitation of speech or body mov...
- Generation of personalized content for harassment, extortion, or intimidation
"GPAIs can be misused for the automated generation of content personalized to target select individuals based on their weak spots [30]. Such attacks may be more efficient and more successful in achiev...
- GPAI assisted impersonation
"GPAI outputs are not always correctly detected as AI-generated across multiple modalities (text, images, audio, video). A malicious actor can use GPAI outputs directly when communicating, or use AI-i...
- AI-driven spear phishing attacks
"Generative models can be misused to target individual users more efficiently by using personalized information [23]. Highly convincing automated fraudulent schemes can exploit the trust of victims by...
- Malicious use and abuse (cybercrime)
"The advanced capabilities and widespread availability of generative AI models make it possible for malicious actors to conduct harmful activities with great efficiency and on a large scale, simultane...
- Malicious use and abuse (sexually explicit content generation)
"An illustrative case of malicious use of generative AI models is the creation of explicit sexual images. Generative AI technologies can be employed to produce deepfakes—for instance, superimposing a...
- Malicious Use of AI
Malicious utilization of AI has the potential to endanger digital security, physical security, and political security. International law enforcement entities grapple with a variety of risks linked to...
- Education - Learning
In contrast to traditional machine learning, the impact of generative AI in the educational sector receives considerable attention in the academic literature. Next to issues stemming from difficulties...
- Writing - Research
Partly overlapping with the discussion on impacts of generative AI on educational institutions, this topic cluster concerns mostly negative effects of LLMs on writing skills and research manuscript co...
- Deception
"AI has become very good at creating fake content. From text to photos, audio and video. The name "Deep Fake" refers to content that is fake at such a level of complexity that our mind rules out the p...
- Nonconsensual use
"Generative AI models might be intentionally used to imitate people through deepfakes by using video, images, audio, or other modalities without their consent."
- Impact on education: plagiarism
"Easy access to high-quality generative models might result in students that use AI models to plagiarize existing work intentionally or unintentionally."
- Impact on education: bypassing learning
"Easy access to high-quality generative models might result in students that use AI models to bypass the learning process."
- Impersonation / identity theft
"Theft of an individual, group or organisation’s identity by a third-party in order to defraud, mock or otherwise harm them or another party"
- IP / copyright / personality / rights loss
"Misuse or abuse of an individual or organisation’s intellectual property, including copyright, trademarks, and patents. & Loss of or restrictions to the rights of an individual to control the commerc...
- Economic manipulation
"Generative AI facilitating targeted manipulation of public opinion for economic purposes (e.g., inflating stock prices)"
- Cheating / plagiarism
"Use of generative AI in an academic setting to either cheat or plagiarize"
- Defamation / libel / slander
"Use of a technology system to create, facilitate or amplify false perception(s) about an individual, group or organisation"
- Sexualization
"The non-consensual sexualisation of an individual or group using a technology or application"
- Social-Engineering
psychologically manipulating victims into performing the desired actions for malicious purposes
- Cybercrime
"The increasingly advanced capabilities and availability of general purpose AI models could be misused for improvements in efficiency and efficacy of cyber crimes. This is especially true for crimes t...
- Misuse tactics that exploit GenAI capabilities (Realistic depiction of human likeness)
-