MIT AI Risk Repository · Risk Sub-Category · 47.02.04
Malicious use and abuse (sexually explicit content generation)
Category: Ethical and social risks
On this page 5 sections
Description
"An illustrative case of malicious use of generative AI models is the creation of explicit sexual images. Generative AI technologies can be employed to produce deepfakes—for instance, superimposing a celebrity’s face onto the body of a performer in an adult film."
From Regulating under Uncertainty: Governance Options for Generative AI (G'sell2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Domain
- 4. Malicious actors
- Causal entity
- Human
- Intent
- Intentional
- Timing
- Post-deployment
Subdomain definition: Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Real-world incidents in this subdomain
- Italian Mediaset Journalist Safiria Leccese's Image Was Reportedly Used in a Purportedly AI-Generated Fake Loan Scam
- Scammers Reportedly Used AI-Cloned Daughter's Voice to Defraud Bay Area Mother in Fake Kidnapping Call
- Guelph, Ontario, Woman Reportedly Lost $14,000 in Purported Deepfake MrBeast Cryptocurrency Scam
- Purportedly AI-Recreated Clips from Beastie Boys' 'Sabotage' Video Reportedly Appeared in FBI Promotional Video Posted by Kash Patel
- Ahmedabad Aadhaar Fraud Racket Reportedly Used Purportedly AI-Generated Deepfakes to Change Businessman's Linked Mobile Number
- Purported Deepfake Video Reportedly Portrayed Nirmala Sitharaman Endorsing Investment Scheme
How other frameworks describe this risk
Other entries from G'sell2024
- Technical and operational risks
- Technical vulnerabilities (Robustness - unexpected behaviour)
- Technical vulnerabilities (Robustness - unexpected behaviour)
- Technical vulnerabilities (Robustness - vulnerability to jailbreaking
- Technical vulnerabilities (Robustness - vulnerability to jailbreaking
- Technical vulnerabilities (The risk of misalignment)
- Technical vulnerabilities (The risk of misalignment)
- Factually incorrect content (inaccuracies and fabricated sources)
- Factually incorrect content (inaccuracies and fabricated sources)
- Opacity (the black box problem)
- Opacity (industry opacity)
- Opacity (industry opacity)