AI incident ·
AI-Enabled Fraud Schemes Reportedly Increasing Consumer Harm and Challenging Detection
In brief
An AI system built by Openai, Ai Tool Creators and 2 others and deployed by Scammers allegedly harmed Bank Customers.
- Risk domain
- Malicious Actors & Misuse
- Occurred
- Coverage
- 1 report
What happened
Scammers are reportedly using AI tools such as language models, voice cloning, and synthetic IDs to create more convincing frauds, leading to financial losses and identity theft. Banks have begun deploying AI-driven verification tools to counter these schemes, but experts warn that AI-enabled fraud continues to cause real-world harm and remains difficult to detect.
Laws that address this harm
Policy angle: Classified under Malicious Actors & Misuse (Fraud, scams, and targeted manipulation) in the MIT AI Risk Repository taxonomy; 5 recorded instruments address this use case.
- India DPDP Act
- Law on Artificial Intelligence (2025)
- Law No. 132/2025 on artificial intelligence
- EU AI Act
- Texas Responsible AI Governance Act (TRAIGA)
Matched from the record's risk domain and country to the instruments recorded here. A reviewer can correct the match in the repository (data/external/incident_overrides.yaml).
News reports (1)
Titles link to the original publisher; report text is not reproduced here.
Who was involved
- Alleged deployer
- Scammers
- Alleged developer
- Openai, Ai Tool Creators, Deepfake Technology Developers, Synthetic Audio Generation Technology Developers
- Alleged harmed party
- Bank Customers
Classification (MIT AI Risk Repository taxonomy)
- Risk domain
- Malicious Actors & Misuse
- Risk subdomain
- 4.3 Fraud, scams, and targeted manipulation
- Causal entity
- Human
- Intent
- Intentional
- Timing
- Post-deployment
- Harm level
- —
- Sectors
- —
- Countries
- —
Risk entries describing this failure mode
Entries from the MIT AI Risk Repository coded to subdomain 4.3.
- Cheating/plagiarism
"Cheating/plagiarism - Use of another person’s or group’s words or ideas without consent and/or acknowledgement."
- IP/copyright loss
"IP/copyright loss - Misuse or abuse of an individual or organisation’s intellectual property, including copyright, trademarks, and patents."
- Financial and business
"Financial and Business - Use or misuse of a technology system in a manner that damages the financial interests of an individual or group, or which causes strategic, operational, legal or financial harm to a business or...
- Impersonation/identity theft
"Impersonation/identity theft - Theft of an individual, group or organisation’s identity by a third-party in order to defraud, mock or otherwise harm them."
- Dehumanisation/objectification
"Dehumanisation/objectification - Use or misuse of a technology system to depict and/or treat people as not human, less than human, or as objects."
- Defamation/libel/slander
"Defamation/libel/slander - Use of a technology system to create, facilitate or amplify false perception(s) about an individual, group, or organisation."
- Misinformation and Manipulation
"Recent studies have demonstrated that LLMs can be exploited to craft deceptive narratives with levels of persuasiveness similar to human-generated content (Pan et al., 2023b; Spitale et al., 2023), to fabri- cate fake n...
- Cybersecurity
"LLMs may exacerbate cybersecurity risks in various ways (Newman, 2024). Firstly, LLMs may significantly amplify the effectiveness of deceptive operations aimed at tricking people into disclosing sensitive information or...
Incidents in the same risk subdomain
- Italian Mediaset Journalist Safiria Leccese's Image Was Reportedly Used in a Purportedly AI-Generated Fake Loan Scam
- Scammers Reportedly Used AI-Cloned Daughter's Voice to Defraud Bay Area Mother in Fake Kidnapping Call
- Guelph, Ontario, Woman Reportedly Lost $14,000 in Purported Deepfake MrBeast Cryptocurrency Scam
- Purportedly AI-Recreated Clips from Beastie Boys' 'Sabotage' Video Reportedly Appeared in FBI Promotional Video Posted by Kash Patel
- Ahmedabad Aadhaar Fraud Racket Reportedly Used Purportedly AI-Generated Deepfakes to Change Businessman's Linked Mobile Number
- ChatGPT Was Reportedly Consulted on Body-Disposal and Concealment Questions Before and After University of South Florida Doctoral Students Were Killed
Other incidents involving Scammers
- Purported Deepfake WhatsApp Scam Impersonating Tamil Nadu Chief Minister and Actor C. Joseph Vijay Reportedly Defrauded Bengaluru Security Guard of ₹104,000 (~$1,085.40)
- Northern Ireland Victim Reportedly Lost £250,000 in Investment Scam Using Purportedly AI-Generated Video of Unnamed Financial Personality
- Hong Kong Man Reportedly Lost More Than HK$10 Million in WhatsApp Scam Using Purported AI-Generated Voice Impersonation of His Father
- Purportedly AI-Powered KT Platform Reportedly Used in Forced Scamming Operation Involving Trafficked Worker in Myanmar
- Italian Mediaset Journalist Safiria Leccese's Image Was Reportedly Used in a Purportedly AI-Generated Fake Loan Scam
- Lyft Driver in Boca Raton, Florida, Allegedly Used Purported Google Gemini-Generated Image to Support False $75 Damage Fee
Source record: incident #735 on the AI Incident Database · all 1 report