AI incident #848 ·
High School Student in Córdoba, Argentina Accused of Using AI to Generate Explicit Images of Classmates
What happened
An 18-year-old Argentine student at the pre-university institute of Manuel Belgrano of Córdoba allegedly used AI tools to generate explicit fake images of at least 22 female classmates by combining their faces with other bodies. These images, posted on pornography websites, included the victims' names, leading to harassment and significant psychological harm. Legal authorities charged the student with serious injuries aggravated by gender violence.
Editor's notes (AI Incident Database)
Reconstructing the timeline of events: (1) Mid-2024: Two female students at the Manuel Belgrano Institute began receiving suspicious Instagram messages from men aged 40–50. (2) July 2024: Victims discovered explicit AI-generated images of themselves on pornographic websites, featuring their names and altered appearances. (3) August 2024: Families of victims reported the student to authorities, who launched an investigation and conducted a search of the accused's home. (4) October 18, 2024: Charges were filed against the accused for gender-based violence and psychological harm; legal experts advocated for updated laws addressing AI misuse. This fourth date is being marked as the incident date, even though the initial events are reported to have begun sometime in the middle of 2024.
Only the incident metadata is stored here. The underlying news reports are on the AI Incident Database (CC BY-SA 4.0); use the links above to read them.
News reports (1)
Coverage catalogued by the AI Incident Database. Titles link to the original publisher; the text is not reproduced here.
Who was involved
- Alleged deployer
- Unnamed 18-year-old Manuel Belgrano male student University students Synthetic media creators Students Men and boys Men Deepfake creators
- Alleged developer
- Synthetic media generation technology developers Synthetic image generation technology developers Deepfake technology developers
- Alleged harmed party
- Women and girls Women Unnamed Manuel Belgrano female students University students Students Privacy Epistemic integrity Educational communities
AI systems implicated
Synthetic media generation technologySynthetic image generation technologyDeepfake technology
Classification (MIT AI Risk Repository taxonomy)
- Risk domain
- Malicious Actors & Misuse
- Risk subdomain
- 4.3 Fraud, scams, and targeted manipulation
- Causal entity
- Human
- Intent
- Intentional
- Timing
- Post-deployment
- Harm level
- —
- Sectors
- —
- Countries
- —
Risk entries describing this failure mode
Entries from the MIT AI Risk Repository coded to subdomain 4.3.
- Impersonation/identity theft
"Impersonation/identity theft - Theft of an individual, group or organisation’s identity by a third-party in order to defraud, mock or otherwise harm them."
- IP/copyright loss
"IP/copyright loss - Misuse or abuse of an individual or organisation’s intellectual property, including copyright, trademarks, and patents."
- Dehumanisation/objectification
"Dehumanisation/objectification - Use or misuse of a technology system to depict and/or treat people as not human, less than human, or as objects."
- Defamation/libel/slander
"Defamation/libel/slander - Use of a technology system to create, facilitate or amplify false perception(s) about an individual, group, or organisation."
- Financial and business
"Financial and Business - Use or misuse of a technology system in a manner that damages the financial interests of an individual or group, or which causes strategic, operational, legal or financial harm to a business or...
- Cheating/plagiarism
"Cheating/plagiarism - Use of another person’s or group’s words or ideas without consent and/or acknowledgement."
- Cybersecurity
"LLMs may exacerbate cybersecurity risks in various ways (Newman, 2024). Firstly, LLMs may significantly amplify the effectiveness of deceptive operations aimed at tricking people into disclosing sensitive information or...
- Domain-Specific Misuses
"Improvements in LLMs may exert greater pressure to apply LLMs to various domains, such as health and education (Eloundou et al., 2023). Crude efforts to use LLMs in such domains, however, may incur harm and should be di...
Related incidents on the AI Incident Database
Linked by AIID editors or by its text-similarity model.
- Algorithm Assessing Risk Faced by Victims of Gender Violence Misclassified Low-Risk Cases, Allegedly Leading to Homicide of Women and Children in Spain
- Argentinian City Government Deployed Teenage-Pregnancy Predictive Algorithm Using Invasive Demographic Data
- Student-Developed Facial Recognition App Raised Ethical Concerns
Incidents in the same risk subdomain
- Italian Mediaset Journalist Safiria Leccese's Image Was Reportedly Used in a Purportedly AI-Generated Fake Loan Scam
- Scammers Reportedly Used AI-Cloned Daughter's Voice to Defraud Bay Area Mother in Fake Kidnapping Call
- Texas Man Arturo Hernandez Allegedly Published AI-Generated Deepfake Pornography Depicting Women in TAKE IT DOWN Act Case
- Guelph, Ontario, Woman Reportedly Lost $14,000 in Purported Deepfake MrBeast Cryptocurrency Scam
- Purportedly AI-Recreated Clips from Beastie Boys' 'Sabotage' Video Reportedly Appeared in FBI Promotional Video Posted by Kash Patel
- Ahmedabad Aadhaar Fraud Racket Reportedly Used Purportedly AI-Generated Deepfakes to Change Businessman's Linked Mobile Number