AI Incident Database
Browse AI incidents
Every incident record (metadata only) from the weekly snapshot of 2026-09-07. Filter, then export the selection with its licence attached; each row links to the full record and its reports.
-
Anysphere AI Support Bot for Cursor Reportedly Invents Login Policy, Leading to Subscription Cancellations AIID ↗
In April 2025, users of Cursor, an AI-powered coding assistant developed by Anysphere, reported being logged out unexpectedly. An AI-powered support bot, "Sam," allegedly responded with an invented login policy to justify the behavior. The hallucinated policy was not based on any real company change. The incident reportedly led to subscription cancellations.
-
Transgender User Alleges ChatGPT Allowed Suicide Letter Without Crisis Intervention AIID ↗
A transgender user, Miranda Jane Ellison, experiencing acute distress reported that ChatGPT (GPT-4) allowed her to write and submit a suicide letter without intervention. The AI is reported to have offered minimal safety language and ultimately acknowledged its failure to act. Ellison reports having been previously flagged for discussing gender and emotional topics. A formal complaint with transcripts was submitted to OpenAI.
-
Springer Nature Book 'Mastering Machine Learning: From Basics to Advanced' Reportedly Published With Numerous Purportedly Nonexistent or Incorrect Citations AIID ↗
"Mastering Machine Learning: From Basics to Advanced," published by Springer Nature in April 2025, reportedly contained numerous purportedly nonexistent or materially incorrect academic citations. Independent checks allegedly found that many referenced works did not exist or were misattributed, with multiple named researchers reportedly confirming they did not author the cited material. The pattern of errors has been described as consistent with known LLM citation hallucinations.
-
ChatGPT Was Alleged to Have Aided Planning of Florida State University Mass Shooting AIID ↗
On April 17, 2025, a shooting at Florida State University killed Tiru Chabba and Robert Morales and injured others. Phoenix Ikner, the accused shooter, has pleaded not guilty. In 2026, victims' families and attorneys reportedly alleged that Ikner used ChatGPT before the attack to discuss mass shootings, firearms, campus activity, and media attention, and that OpenAI failed to flag or escalate the exchanges. OpenAI denied responsibility.
-
Microsoft Reportedly Blocks 1.6 Million Bot Signup Attempts Per Hour Amid Global AI-Driven Fraud Surge AIID ↗
Between April 2024 and April 2025, Microsoft reportedly blocked 1.6 million bot signups per hour and disrupted $4 billion in fraud attempts linked to AI-enhanced scams. The company's Cyber Signals report details how generative AI is being used to fabricate realistic e-commerce sites, job offers, customer service bots, and phishing lures. Fraud actors now automate mass-deceptive campaigns with fake reviews, deepfakes, and cloned brand domains at unprecedented scale and speed.
-
Alleged Deepfake Investment Scam Uses Economist David Rosenberg's Likeness on Meta Platforms AIID ↗
Fraudsters allegedly used AI-generated content and social media ads to impersonate economist David Rosenberg and promote a fraudulent investment scheme. Victims were reportedly directed to WhatsApp groups with fake stock tips, resulting in substantial financial losses, with some exceeding $500,000. The ads reportedly appeared on Meta platforms, including Facebook and Instagram. Rosenberg and his family reported the scam to authorities.
-
$31,000 Sanction in Lacey v. State Farm Tied to Purportedly Undisclosed Use of LLMs and Erroneous Citations AIID ↗
In the case of Lacey v. State Farm, two law firms were sanctioned $31,000 after submitting a legal brief containing reportedly erroneous citations generated using AI tools. The court reportedly found that the lawyers failed to disclose the use of AI, neglected to verify its output, and refiled a revised brief with additional inaccuracies. Judge Michael Wilner deemed the conduct reckless and issued sanctions for what he described as "improper" and "misleading" legal filings.
-
AI-Powered Presentation Tool Gamma Implicated in Multi-Stage Phishing Campaign AIID ↗
Attackers reportedly exploited Gamma, an AI-powered presentation tool, to create convincing presentation pages that hosted links to a spoofed Microsoft SharePoint login portal. The phishing flow allegedly used compromised email accounts, Cloudflare Turnstile for bot evasion, and adversary-in-the-middle (AiTM) tactics to validate credentials in real time and capture session cookies. The campaign aimed to bypass MFA and compromise accounts.
-
Reported Emergence of 'Vegetative Electron Microscopy' in Scientific Papers Traced to Purported AI Training Data Contamination AIID ↗
Researchers reportedly traced the appearance of the nonsensical phrase "vegetative electron microscopy" in scientific papers to contamination in AI training data. Testing indicated that large language models such as GPT-3, GPT-4, and Claude 3.5 may reproduce the term. The error allegedly originated from a digitization mistake that merged unrelated words during scanning, and a later translation error between Farsi and English.
-
Multiple LLMs Allegedly Endorsed Suicide as a Viable Option During Non-Adversarial Mental Health Venting Session AIID ↗
Substack user @interruptingtea reports that during a non-adversarial venting session involving suicidal ideation, multiple large language models (Claude, GPT, and DeepSeek) responded in ways that allegedly normalized or endorsed suicide as a viable option. The user states they were not attempting to jailbreak or manipulate the models, but rather expressing emotional distress. DeepSeek reportedly reversed its safety stance mid-conversation.
-
16-Year-Old Allegedly Received Suicide Method Guidance from ChatGPT Before Death AIID ↗
16-year-old Adam Raine reportedly died by suicide after allegedly confiding in OpenAI's ChatGPT-4o, which he reportedly used extensively in the months prior. Transcripts reportedly show the chatbot provided empathetic support but also allegedly offered details on suicide methods, validated attempts, and at times discouraged disclosure to family. His parents have reportedly filed a wrongful death lawsuit against OpenAI, alleging design choices nurtured dependency and failed safeguards.
-
Reportedly Unsafe Deployment of Llama.cpp Reveals Interactive AI-Generated CSAM Roleplay Prompts AIID ↗
A study by UpGuard reports that misconfigured llama.cpp servers publicly exposed user prompts, including hundreds of interactive roleplay scenarios. Some prompts explicitly described fictional sexual abuse of children aged 7–12. While no real children were involved, the findings demonstrate how open-source LLMs can be exploited to generate AI-enabled child sexual abuse material (CSAM).
-
Norwegian Supreme Court Receives Legal Filing with Fabricated Citations Allegedly Generated by AI Tool AIID ↗
In early 2025, Norway’s Supreme Court (Høyesterett) received a legal filing containing hallucinated legal citations and quotes, allegedly inserted by an attorney using an AI tool without verification. The fabricated sources were detected during standard review. While no sanctions were issued, the Court updated its guidance to lawyers, warning that AI use does not exempt them from professional responsibility.
-
Alleged AI-Generated Clone of Exante Brokerage Used to Defraud U.S. Investor via JPMorgan Account AIID ↗
Scammers used AI tools to clone the broker Exante and defraud at least one U.S. victim by registering a JPMorgan Chase account and replicating Exante’s trading interface. AI-generated fake documents, deepfakes, and cloned websites enabled the scheme. Exante, which does not operate in the U.S., confirmed the fraud and filed reports with multiple U.S. agencies.
-
Jailbroken Lovable AI Allegedly Used to Generate and Host Phishing Pages, Steal Credentials, and Bypass Security AIID ↗
The generative AI platform Lovable, which is used for building web apps, was reportedly jailbroken to create and host full phishing campaigns. These campaigns allegedly included credential-harvesting login pages, evasion techniques, and real-time exfiltration via services like Telegram and Firebase. The AI system was reportedly used for generating the phishing content and and then deploying it live on a URL hosted under its own subdomain.
-
Jiushi Autonomous Delivery Vehicle Reportedly Dragged Fallen Electric Scooter During Delivery Run in Xianyang, Shaanxi AIID ↗
A Jiushi autonomous delivery vehicle reportedly operating on a delivery route in Xianyang, Shaanxi, was filmed dragging a fallen electric scooter that had allegedly been left in the road after an earlier traffic incident. Company statements said the vehicle attempted to avoid the scooter but failed to fully recognize its shape and position, causing part of it to become caught under a wheel. No injuries were reported, and the vehicle was reportedly suspended pending investigation.
-
Malaysian Teenager Allegedly Arrested for Creating and Selling AI-Generated Deepfake Images of Schoolmates and Alumni in Johor AIID ↗
Authorities in Johor, Malaysia reportedly arrested a 16-year-old student who allegedly used AI-enabled tools to generate and sell deepfake images depicting schoolmates and former students. According to police, the images were created from social media photos and distributed online for payment. The alleged activity affected dozens of victims, including minors, some of whom may have been unaware of the misuse.
-
Reported AI-Generated Video Purportedly Depicting Keanu Reeves Debating Elon Musk Circulates as Misinformation on Social Media AIID ↗
In April 2025, a video circulated online reportedly depicting a televised debate between Keanu Reeves and Elon Musk about AI. The video, posted by the YouTube channel Voices of Change, purportedly used AI-generated voice and visual content to simulate the event. Independent analysis found signs of deepfake audio and imagery, and no credible evidence supported the event's existence.
-
Deepfake Job Applicant Allegedly Used AI Tools to Apply for Remote Role at U.S. Security Startup AIID ↗
Voice authentication startup Pindrop Security reported a job candidate used deepfake software and other AI tools in an attempted scam. This is part of a growing trend in international scammers using AI tools to apply for remote US-based jobs, sometimes successfully.
-
Purported Deepfake Depicts Altercation Between Bougainville President Ishmael Toroama and Papua New Guinea Prime Minister James Marape AIID ↗
A purported AI‑generated deepfake video circulated online falsely depicting a physical altercation between Bougainville President Ishmael Toroama and Papua New Guinea Prime Minister James Marape. The footage reportedly caused public confusion and risked damaging relations between Bougainville and Papua New Guinea. Toroama reportedly condemned the video as false and malicious, as well as potentially destabilizing, urging stronger action against misleading AI‑generated content.
-
Hong Kong Syndicate Allegedly Used AI-Generated Facial Composites to Open Bank Accounts AIID ↗
Hong Kong police reportedly arrested eight individuals accused of using purportedly AI-generated facial composites to open bank accounts with altered ID photos. Of 44 applications, 30 reportedly succeeded after passing online identity checks. The accounts were then allegedly used to apply for loans and make credit card purchases totaling HK$860,000, and to launder over HK$1.2 million in suspected criminal proceeds. Police linked the operation to local triad-affiliated fraud networks.
-
Alleged Deepfake Investment Scam in Spain Defrauds 208 Victims of €19 million ($20.9 million) AIID ↗
Spanish police arrested six individuals allegedly behind a €19M ($20.9M) global investment scam powered by AI. The operation used deepfake ads featuring national celebrities to deceive victims, many of whom were selected through targeting algorithms. Scammers posed as financial advisors and fake officials, cycling through romance baiting, investment fraud, and recovery scams. AI-generated content amplified trust and engagement.
-
Reported Darknet Launch of Xanthorox AI Introduces Autonomous Cyberattack Platform AIID ↗
Xanthorox AI is a malicious, modular AI system released on darknet forums in early 2025. Designed from scratch for offensive cyber operations, it runs on private infrastructure and includes models for code generation, phishing, malware, social engineering, and real-time voice/image input. Its release represents a deliberate deployment of an autonomous attack platform.
-
Deepfake of Bermuda Premier David Burt Promotes Investment Scam Using Royal Gazette Branding AIID ↗
The Government of Bermuda warned the public about a deepfake video circulating on Facebook that falsely portrayed Premier David Burt promoting a government investment scheme. The deepfake video reportedly mimicked the branding of The Royal Gazette and was linked to fraudulent social media accounts. Authorities confirmed the content was fake and urged the public to report such scams and to be vigilant about them.
-
Essex Man Sentenced to Five Years in Prison for Having Generated and Shared Deepfake Pornography of at Least 20 Women and a Minor AIID ↗
Between March 2023 and May 2024, Brandon Tyler of Essex used AI to generate explicit deepfake pornography of at least 20 women he knew personally, including a 16-year-old. He manipulated their social media photos and shared them, along with their personal details, in online forums promoting sexual violence. He was sentenced to five years in prison in April 2025 under the UK's criminal law against sharing sexually explicit deepfakes, in what is one of the first major prosecutions of its kind.
-
Baltimore Lawsuit Alleged DraftKings and FanDuel Used Machine-Learning-Driven Targeting to Exploit Vulnerable Gamblers AIID ↗
In Baltimore, the city sued DraftKings and FanDuel, alleging that the companies used predictive modeling with machine-learning algorithms, extensive user data, personalized promotions, push notifications, and VIP programs to identify and exploit vulnerable bettors, including people showing signs of gambling disorder. The complaint alleges these systems were designed to maximize user lifetime value and contributed to financial and public-health harms among Baltimore residents.
-
AI Voice Clone of Texas Woman Used in Distress Scam Targeting Brother in Port Neches AIID ↗
Jace Edgar of Port Neches, Texas reportedly received a scam phone call using AI-generated voice cloning to mimic his sister in distress. Believing she had been in an accident, Edgar began to act before noticing the caller avoided direct questions. When he contacted his sister, she was unharmed and unaware of the call. Authorities confirmed this was part of a broader rise in AI-powered scams, urging the public to stay vigilant and avoid sharing personal info with unknown callers.
-
DOGE Reportedly Relied on Unvetted ChatGPT Outputs in Canceling National Endowment for the Humanities Grants AIID ↗
Department of Government Efficiency (DOGE) staff reportedly fed National Endowment for the Humanities grant descriptions into ChatGPT to determine whether projects were "DEI," then allegedly used those outputs to help compile a list of grants to terminate. Beginning in April 2025, NEH canceled awards and clawed back funding, reportedly disrupting humanities organizations and projects nationwide after officials allegedly relied on a purportedly flawed, insufficiently vetted AI screening process.
-
Reported AI‑Generated Deepfake Impersonations of Public Figures Allegedly Used in Coordinated Stock Pump‑and‑Dump Scheme Targeting Israeli Investors AIID ↗
A coordinated fraud campaign in Israel allegedly used AI‑generated deepfake impersonations of public figures including Amir Yaron, Benjamin Netanyahu, Eyal Golan, Noa Kirel, Gal Gadot, Elon Musk, and Mark Zuckerberg to promote NASDAQ‑listed Ostin Technology Group Co. Ltd. (OST) in a pump‑and‑dump scheme. Paid social media ads, WhatsApp groups, and fake investment platforms reportedly lured investors before the stock collapsed in early July 2025, causing major reported losses.
-
Reportedly AI-Generated Image Circulates Amid Reports of Tanzania Revenue Authority Job Interviews AIID ↗
An image circulated on Facebook purporting to show hundreds of Tanzanians lining up for just five jobs at the Tanzania Revenue Authority (TRA). The photo, reportedly AI-generated, gained traction shortly after TRA invited over 100,000 applicants for interviews on March 29–30. Though TRA had announced more than 1,500 openings, the image misrepresented the situation, stoking public anxiety during a period of high unemployment.
-
OpenAI's 4o Model Allegedly Used to Generate Fake Receipts and Prescriptions AIID ↗
OpenAI's new image generator, 4o, was reportedly used to produce realistic fraudulent documents, including fake restaurant receipts, prescriptions for controlled substances, and potentially other identity or financial documents. Users demonstrated how the model could be prompted to bypass safety restrictions to be misused for expense fraud, medical forgery, and other various forms of document falsification.
-
GenNomis AI Database Reportedly Exposes Nearly 100,000 Deepfake and Nudify Images in Public Breach AIID ↗
In March 2025, cybersecurity researcher Jeremiah Fowler discovered an unprotected database linked to GenNomis by AI-NOMIS, a South Korean company offering face-swapping and "nudify" AI services. The exposed 47.8GB dataset included nearly 100,000 files. Many depicted explicit deepfake images, some involving minors or celebrities. No personal data was found, but the breach was a serious failure in data security and consent safeguards in AI image-generation platforms.
-
GPT-4o Allegedly Reinforced California Man Michael Lines's Delusions Before Suicide Attempt AIID ↗
In March 2025, Michael Lines, a 34-year-old California man, allegedly spent weeks discussing religious delusions with GPT-4o after disclosing his bipolar disorder and medication use. His complaint says the chatbot validated his belief that he was Jesus, adopted a divine persona, and failed to redirect him when he expressed suicidal intent. Lines later overdosed and survived after law enforcement found him.
-
Reported Use of AI Apps to Create Sexualized Deepfake Images of High School Students at Cascade High School in Iowa AIID ↗
Students of the Western Dubuque Community School District in Iowa reported that classmates allegedly used AI-enabled applications to generate sexualized deepfake images depicting fellow students. According to local reporting and the Dubuque County Sheriff's Office, the fabricated images were created and shared without consent. The school district reportedly held a public session to address policy gaps and student support following the incident.
-
Purported Meta AI Chatbot Persona 'Big sis Billie' Reportedly Engages in Romantic Roleplay and Provides Address, Linked to User's Fatal Fall AIID ↗
Reuters reported that a cognitively impaired New Jersey man, Thongbue Wongbandue, died after rushing to meet "Big sis Billie," a purported Meta AI chatbot on Facebook Messenger that allegedly assured him it was real, expressed romantic interest, and invited him to a New York apartment. The family and transcripts reportedly suggest the chatbot's outputs contributed to his decision to travel, allegedly resulting in a fatal fall.
-
Fact-Checking Finds Reportedly AI-Generated Video Misattributed Hypertension Cure Endorsements to Taiwo Ajai-Lycett and Chinonso Egemba AIID ↗
Fact-checking reports that a purported AI-generated video circulated on Facebook depicting veteran actor Taiwo Ajai-Lycett discussing a hypertension diagnosis and promoting an unproven treatment by influencer Chinonso Egemba. According to the report, the video exhibited signs of AI generation, including unnatural speech and mismatched lip movements. Both Ajai-Lycett and Egemba have denied involvement, and no credible medical evidence supports the claimed cure.
-
Manipulated Video Using AI-Generated Audio Targets Pakistan's Prime Minister Shehbaz Sharif with Fabricated Interview AIID ↗
A viral video falsely claimed to show Layal al-Ekhtiar of Al Arabiya confronting Pakistan's Prime Minister Shehbaz Sharif over funding a 30-person Umrah delegation with public money while seeking aid from Saudi Arabia. Investigators concluded the video had been dubbed with AI-generated audio. The original interview was broadcast by Saudi-owned Al Arabiya in January 2023, and it contained no such exchange. Lip-sync inconsistencies reportedly confirmed the manipulation.
-
Sora Video Generator Has Reportedly Been Creating Biased Human Representations Across Race, Gender, and Disability AIID ↗
A WIRED investigation found that OpenAI's video generation model, Sora, exhibits representational bias across race, gender, body type, and disability. In tests using 250 prompts, Sora was more likely to depict CEOs and professors as men, flight attendants and childcare workers as women, and showed limited or stereotypical portrayals of disabled individuals and people with larger bodies.
-
Purported DOGE Contract Review Tool Cited in Reports of AI-Driven Misjudgments in VA Budget Cuts AIID ↗
A purported AI tool using LLMs was deployed to classify Veterans Affairs contracts as expendable based on limited text and simplified criteria. The system allegedly produced hallucinated values and flagged critical healthcare and research services for cancellation. Reportedly, at least two dozen flagged contracts were later terminated.
-
Alleged Fraudulent Prompts via AIXBT Dashboard Led Purported AI Trading Agent to Transfer 55.5 ETH from Simulacrum Wallet AIID ↗
A reported hacker attack allegedly compromised the autonomous AI crypto bot AIXBT, purportedly resulting in the theft of 55.5 ETH (approximately $106,200). The attacker is reported to have infiltrated the secure dashboard of the AIXBT autonomous system at 2:00 AM UTC on March 18, 2025, and allegedly queued two fraudulent prompts that instructed the AI agent to transfer funds from its simulacrum wallet.
-
AI-Enabled Organized Crime Expands Across Europe AIID ↗
Europol’s EU Serious and Organised Crime Threat Assessment (EU-SOCTA) 2025 warns that AI is accelerating the growth of organized crime throughout Europe. Criminal networks are leveraging AI for cyber fraud, ransomware, money laundering, and child exploitation, while AI-powered social engineering and automation are making criminal operations more scalable and harder to detect.
-
LLM Scrapers Allegedly Target Multiple Open Source Projects Disrupting the FOSS Ecosystem AIID ↗
In mid-March 2025, KDE's GitLab infrastructure was reportedly disrupted by aggressive AI web scrapers originating from Alibaba IP ranges. These bots allegedly ignored robots.txt and spoofed browser headers, which in turn purportedly overwhelmed the site and caused outages for developers. Similar incidents reportedly affected other FOSS projects like GNOME, SourceHut, and Fedora. The scraping is allegedly tied to large language model training, and reportedly imposes real costs and delays.
-
Alleged Use of AI to Create Sexualized Deepfake Images of Middle School Students Under Investigation in Bucks County, Pennsylvania AIID ↗
Authorities in Bucks County, Pennsylvania reported that a middle school student allegedly used AI-enabled tools to create sexualized deepfake images depicting classmates. The incident, which reportedly occurred in March 2025, involved images generated without consent and is under investigation by local police and the district attorney's office. The alleged conduct may fall under a new Pennsylvania law addressing non-consensual AI-generated sexual imagery, particularly involving minors.
-
Alleged AI-Generated IRS Scam Websites Used to Defraud U.S. Taxpayers AIID ↗
Scammers have allegedly been using AI-generated imposter websites and phishing emails to impersonate the IRS. They have reportedly been tricking taxpayers into providing personal and financial information. There has been a reported surge in tax-related AI scams leading up to Tax Day 2025, with fraudulent domains mimicking IRS services, along with fake websites, emails, and text messages. The IRS has warned taxpayers to verify official sites and avoid unsolicited links.
-
Docomo Pacific CEO Reports Mother Targeted by Purported AI-Enabled Scam in Guam AIID ↗
Docomo Pacific's CEO, Christine Baleto, disclosed that her elderly mother was targeted by an alleged AI-enabled scam call on Guam. The scammer reportedly posed as a federal agent and used personal details and intimidation, demanding sensitive information under false pretenses. The company's advisory suggests AI voice cloning may have been involved, warning that such technology can convincingly mimic family members' voices, increasing risk for seniors on the island.
-
AI-Generated Voice Cloning of Trump Falsely Proposes Renaming the District of Columbia to the District of America AIID ↗
A viral audio clip circulating on TikTok, Facebook, X, and other platforms falsely portrayed Donald Trump saying he would rename Washington, D.C. as the "District of America." The voice was confirmed to be AI-generated by digital forensics experts. It was initially posted by a now-deleted TikTok account. The clip is reported to have spread rapidly across political lines before it was debunked.
-
Scammers Reportedly Using Deepfake Video Calls to Impersonate Executives in Singapore and Orchestrate Corporate Bank Transfers AIID ↗
Scammers in Singapore are reportedly using AI-generated deepfake video calls to impersonate corporate executives. The calls seek to deceive employees into authorizing fraudulent bank transfers. Usually, it is reported, victims will receive WhatsApp messages inviting them to Zoom meetings. During the calls, the scammers are disguised as senior leaders and will instruct the employees to transfer company funds for fake business transactions and to disclose personal data.
-
Alleged ChatGPT Misuse by Contractor Leads to Reported Data Exposure in New South Wales Resilient Homes Program AIID ↗
A former contractor of the New South Wales Reconstruction Authority reportedly uploaded a spreadsheet containing personal and health information of Resilient Homes Program applicants to ChatGPT during a three-day period in March 2025. Up to 3,000 people may have reportedly been affected.
-
Misleading Deepfake Video of Trump Mocking Zelenskyy for Clothing Goes Viral on Facebook AIID ↗
In March 2025, an AI-generated deepfake video of Donald Trump criticizing Ukrainian President Volodymyr Zelenskyy for his clothing circulated widely on Facebook. In the video, Trump mocks Zelenskyy by calling him "Temu Zelenskyy." The video was created by a parody YouTube channel using AI voice and face synthesis tools, but it was reportedly mistaken by some viewers as real.
-
Attackers Reportedly Deployed Simulated AI Support Chatbot to Trick Instagram Business Users into Adding Malicious 2FA Login AIID ↗
A phishing campaign has reportedly been impersonating Meta support using a fake chatbot interface to hijack Instagram Business accounts. Victims received emails claiming ad violations and were directed to a fraudulent site mimicking Meta's support. There, a simulated chatbot instructed users to add the attacker’s Authenticator app as a secure login method, enabling account takeover. It remains unclear whether the chatbot used AI or was human-operated via a bot-like interface. See editor's note.
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.